[arch-general] scripts to analyze and manage /var/cache/pacman/pkg for installed/uninstalled & gen install scr for uninstalled
Guys, With a number of arch boxes now, I routinely rsync the package cache from box to box for updates, etc. Today what started out as a "1-liner" to show what scripts in /var/cache/pacman/pkg were installed, uninstalled (what I called orphaned) and then to create summary files and create a script to install the orphans -- well -- turned into somewhat of a longer script that I thought I would pass along. The script 'chkinstpkgs.sh' does just that. Check the installed packages and give you a summary of what is going on with your package cache. I'm sure I just reinvented the wheel, but it was fun nonetheless. The script is at: http://www.3111skyline.com/download/Archlinux/scripts/chkinstpkgs.sh I left all my development commented out at the end of the file -- there were some neat parse parameter substitutions, etc.. that I ultimately didn't use that might be helpful for anyone wanting to mess with managing their package cache, or at least provide a "go-by" for anyone interested in learning a bit more bash. The -h | --help option displays a short help: 18:30 alchemy:~/scr/arch> sh chkinstpkgs.sh -h Usage: chkinstpkgs.sh [package-dir] [orphans-file] [installed-file] [install-orphans-file] chkinstpkgs.sh checks all packages in 'package-dir' to see whether the file is installed or whether the package is an orphan (exists in 'package-dir' but isn't installed). Results are printed to the screen and saved in the users home directory as shown below: Where the defaults are: package-dir=/var/cache/pacman/pkg/ orphans-file=~/pkgs-orphans.txt installed-file=~/pkgs-installed.txt install-orphans-file=~/pkgs-install-orphans.sh Options: -h | --help display usage help and exit -s | --summary display packages totals, installed, orphaned and exit The -s | --summary just gives a numeric summary of the installed/orphaned: 18:30 alchemy:~/scr/arch> sh chkinstpkgs.sh -s Determining installed/orphaned packages in /var/cache/pacman/pkg... Initializing the package-information arrays Checking installed packages Total Number of Packages : 2072 Total Number Installed : 1626 Total Number Orphaned : 446 Just running the whole script produces a complete summary on the screen and also creates the summary files shown above under the defaults: 18:31 alchemy:~/scr/arch> head ~/pkgs-orphans.txt error: package "a2ps" not found error: package "acct" not found error: package "acpi" not found error: package "acpid" not found <snip> 18:42 alchemy:~/scr/arch> head ~/pkgs-installed.txt Installed Packages - 12/07/09 a2ps-4.14-1-x86_64.pkg.tar.gz a52dec-0.7.4-4-x86_64.pkg.tar.gz aalib-1.4rc5-6-x86_64.pkg.tar.gz abs-2.3.3-1-x86_64.pkg.tar.gz acct-6.3.2-2-x86_64.pkg.tar.gz <snip> 18:42 alchemy:~/scr/arch> head ~/pkgs-install-orphans.sh pacman -Sy a2ps acct acpi acpid acpitool acpitool agg archlinux-menus archlinux-menus b43-fwcutter bc bind blas bridge-utils capi4k-utils cyrus-sasl cyrus-sasl-plugins db4.5 dbus-qt3 ddcxinfo-arch dhcp dmapi dmraid dovecot dvd-slideshow filelight flashplugin font-bitstream-speedo font-bitstream-speedo foomatic-db foomatic-db-nonfree <snip> Just delete the ones you don't want to install (beats creating the list from scratch) The script takes ~ 30 seconds per 1000 packages to run. It took about a minute to complete on my laptop. Enjoy. Other package cache scripts available: http://www.3111skyline.com/download/Archlinux/scripts/fduparchpkg http://www.3111skyline.com/download/Archlinux/scripts/fduppkg The above scripts remove duplicate packages from the package cache and move them to /home/backup/pkg-old and /home/backup/pkg-older. The workhorse is fduppkg and fduparchpkg is just a wrapper that calls fduppkg. I just create links to the files in /usr/local/bin: lrwxrwxrwx 1 root root 30 Dec 7 18:22 fduparch -> /home/david/scr/pm/fduparchpkg lrwxrwxrwx 1 root root 26 Nov 5 02:21 fduppkg -> /home/david/scr/pm/fduppkg then just call fduparch and it will move all duplicates from /var/cache/pacman/pkg to /home/backup/pkg-old and then remove all duplicates (newly created) from /home/backup/pkg-old and move them to /home/backup/pkg-older. That way you always have current packages in /var/cache/pacman/pkg, a clean set of the next most recent packages in /home/backup/pkg-old and the package dump of /home/backup/pkg-older where you can keep as many as you want. One primary benefit of moving the packages to /home/backup is it frees up space on your / partition as well as cleaning up /var/cache/pacman. I have posted the dup scripts before, but these have a few updates that optimize things better to cut down on runtime. Even with several thousand packages, the scripts run in about a minute as well. Last note: for any Archers who would like to dig into bash scripting a little more. I like to think these scripts have a lot of cool stuff in them (at least I think so...) and provide a good reference for loops, arrays, array and loop index manipulation for optimizations, redirection, subshells, functions, parameter substitution and substring extraction that are fairly good tools to put in your *nx toolbox. Believe it or not, I actually take some time to comment the scripts with that purpose in mind. (and so I can remember what the hell I was thinking a year from now...) For all you gurus out there, if you happen to look at the scripts and see things I could do better, I welcome the feedback because I'm always open to adding new tools to my toolbox as well ;-) -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com
In the event of a sudden termination event for the program, it would
be nice to have a check there.
Look into "trap" and make sure no matter what your IFS gets set back
to it's original value
even if someone does a CTRL+ALT+DEL. ;)
Dave Crouse
On Mon, Dec 7, 2009 at 7:02 PM, David C. Rankin
Guys,
With a number of arch boxes now, I routinely rsync the package cache from box to box for updates, etc. Today what started out as a "1-liner" to show what scripts in /var/cache/pacman/pkg were installed, uninstalled (what I called orphaned) and then to create summary files and create a script to install the orphans -- well -- turned into somewhat of a longer script that I thought I would pass along.
The script 'chkinstpkgs.sh' does just that. Check the installed packages and give you a summary of what is going on with your package cache. I'm sure I just reinvented the wheel, but it was fun nonetheless. The script is at:
http://www.3111skyline.com/download/Archlinux/scripts/chkinstpkgs.sh
I left all my development commented out at the end of the file -- there were some neat parse parameter substitutions, etc.. that I ultimately didn't use that might be helpful for anyone wanting to mess with managing their package cache, or at least provide a "go-by" for anyone interested in learning a bit more bash.
The -h | --help option displays a short help:
18:30 alchemy:~/scr/arch> sh chkinstpkgs.sh -h
Usage: chkinstpkgs.sh [package-dir] [orphans-file] [installed-file] [install-orphans-file]
chkinstpkgs.sh checks all packages in 'package-dir' to see whether the file is installed or whether the package is an orphan (exists in 'package-dir' but isn't installed). Results are printed to the screen and saved in the users home directory as shown below:
Where the defaults are: package-dir=/var/cache/pacman/pkg/ orphans-file=~/pkgs-orphans.txt installed-file=~/pkgs-installed.txt install-orphans-file=~/pkgs-install-orphans.sh
Options: -h | --help display usage help and exit -s | --summary display packages totals, installed, orphaned and exit
The -s | --summary just gives a numeric summary of the installed/orphaned:
18:30 alchemy:~/scr/arch> sh chkinstpkgs.sh -s
Determining installed/orphaned packages in /var/cache/pacman/pkg...
Initializing the package-information arrays Checking installed packages
Total Number of Packages : 2072 Total Number Installed : 1626 Total Number Orphaned : 446
Just running the whole script produces a complete summary on the screen and also creates the summary files shown above under the defaults:
18:31 alchemy:~/scr/arch> head ~/pkgs-orphans.txt error: package "a2ps" not found error: package "acct" not found error: package "acpi" not found error: package "acpid" not found <snip>
18:42 alchemy:~/scr/arch> head ~/pkgs-installed.txt
Installed Packages - 12/07/09
a2ps-4.14-1-x86_64.pkg.tar.gz a52dec-0.7.4-4-x86_64.pkg.tar.gz aalib-1.4rc5-6-x86_64.pkg.tar.gz abs-2.3.3-1-x86_64.pkg.tar.gz acct-6.3.2-2-x86_64.pkg.tar.gz <snip>
18:42 alchemy:~/scr/arch> head ~/pkgs-install-orphans.sh pacman -Sy a2ps acct acpi acpid acpitool acpitool agg archlinux-menus archlinux-menus b43-fwcutter bc bind blas bridge-utils capi4k-utils cyrus-sasl cyrus-sasl-plugins db4.5 dbus-qt3 ddcxinfo-arch dhcp dmapi dmraid dovecot dvd-slideshow filelight flashplugin font-bitstream-speedo font-bitstream-speedo foomatic-db foomatic-db-nonfree <snip>
Just delete the ones you don't want to install (beats creating the list from scratch)
The script takes ~ 30 seconds per 1000 packages to run. It took about a minute to complete on my laptop. Enjoy.
Other package cache scripts available:
http://www.3111skyline.com/download/Archlinux/scripts/fduparchpkg http://www.3111skyline.com/download/Archlinux/scripts/fduppkg
The above scripts remove duplicate packages from the package cache and move them to /home/backup/pkg-old and /home/backup/pkg-older. The workhorse is fduppkg and fduparchpkg is just a wrapper that calls fduppkg. I just create links to the files in /usr/local/bin:
lrwxrwxrwx 1 root root 30 Dec 7 18:22 fduparch -> /home/david/scr/pm/fduparchpkg lrwxrwxrwx 1 root root 26 Nov 5 02:21 fduppkg -> /home/david/scr/pm/fduppkg
then just call fduparch and it will move all duplicates from /var/cache/pacman/pkg to /home/backup/pkg-old and then remove all duplicates (newly created) from /home/backup/pkg-old and move them to /home/backup/pkg-older. That way you always have current packages in /var/cache/pacman/pkg, a clean set of the next most recent packages in /home/backup/pkg-old and the package dump of /home/backup/pkg-older where you can keep as many as you want.
One primary benefit of moving the packages to /home/backup is it frees up space on your / partition as well as cleaning up /var/cache/pacman. I have posted the dup scripts before, but these have a few updates that optimize things better to cut down on runtime. Even with several thousand packages, the scripts run in about a minute as well.
Last note: for any Archers who would like to dig into bash scripting a little more. I like to think these scripts have a lot of cool stuff in them (at least I think so...) and provide a good reference for loops, arrays, array and loop index manipulation for optimizations, redirection, subshells, functions, parameter substitution and substring extraction that are fairly good tools to put in your *nx toolbox. Believe it or not, I actually take some time to comment the scripts with that purpose in mind. (and so I can remember what the hell I was thinking a year from now...)
For all you gurus out there, if you happen to look at the scripts and see things I could do better, I welcome the feedback because I'm always open to adding new tools to my toolbox as well ;-)
-- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com
Crouse wrote:
In the event of a sudden termination event for the program, it would be nice to have a check there. Look into "trap" and make sure no matter what your IFS gets set back to it's original value even if someone does a CTRL+ALT+DEL. ;)
Dave Crouse
Dave, Thank you. I know trap. There is no need to reset IFS when run in a script (see my comments in the script, "It's just good form". Why? When you execute a script, it gets its own environment (a subshell so to speak) and nothing that happens in a subshell can effect the current environment. For example, do this: 03:15 alchemy:~> echo "'$IFS'" ' ' (That's ' \t\n' [space,tab,newline] by default) Now let's change IFS in a subshell [ any command executed enclosed in parenthesis on the command line gets its own subshell ], confirm the change, then check the current environment: 22:44 alchemy:~> ( IFS='$__'; echo "'$IFS'" ); echo "'$IFS'" '$__' ' ' You can see that the IFS set and echoed in the subshell remained the $__ that was assigned, but the second echo of IFS after the subshell terminated is still the familiar old ' \t\n', That's when you don't need to reset it in a script, but that is also why I do it anyway in the scripts and I write the comments in the scripts telling you it isn't necessary but to do it anyway as a matter of good form. Very few understand the subshell rule, but if you always reset IFS, you will always be OK :p Thanks for making me remember it though! -- David C. Rankin, J.D.,P.E. Rankin Law Firm, PLLC 510 Ochiltree Street Nacogdoches, Texas 75961 Telephone: (936) 715-9333 Facsimile: (936) 715-9339 www.rankinlawfirm.com
participants (2)
-
Crouse
-
David C. Rankin