[pacman-dev] [PATCH 0/5] Version the local database
With removing directory symlink support, we need to make an adjustment to the users local database to adjust any path that relied in that "feature". As this is not a readily detectable change, as was the case in our last local database adjustment, we need to add something to track the local database version. The file ".alpm_db_version" is added to the local database root. This file is automatically created whenever the local database directory is absent or empty. This store the current database version as an integer. I have chosen to use the (next) libalpm soname version so it is easy for us to track what versions are needed. Allan McRae (5): Create local database directory if it is missing Add version file when creating local database directory Add version file to empty local database Check the version of the local database during validation Extend database upgrade script to handle alpm db version 9 lib/libalpm/be_local.c | 118 +++++++++++++++++++++++++++------------- scripts/pacman-db-upgrade.sh.in | 46 ++++++++++++---- test/pacman/pmdb.py | 2 + test/pacman/pmtest.py | 3 + 4 files changed, 121 insertions(+), 48 deletions(-) -- 1.8.3.3
This means that a missing local database becomes an error (as it
should be immediately created). Note this only creates the "local"
directory and not its parent, which is check for during locking.
Signed-off-by: Allan McRae
The version of the local pacman database is stored in its root in the file
.alpm_db_version. The version is starting at 9, corresponding to the
next libalpm library version.
Signed-off-by: Allan McRae
If a user manually creates the local database directory, or has an empty
local database for some other reason, we silently add a version file
Signed-off-by: Allan McRae
Signed-off-by: Allan McRae
Signed-off-by: Allan McRae
On Tue, Jul 16, 2013 at 8:22 PM, Allan McRae
Signed-off-by: Allan McRae
--- scripts/pacman-db-upgrade.sh.in | 46 ++++++++++++++++++++++++++++++++--------- 1 file changed, 36 insertions(+), 10 deletions(-) diff --git a/scripts/pacman-db-upgrade.sh.in b/scripts/pacman-db-upgrade.sh.in index a1630c5..d2cd78d 100644 --- a/scripts/pacman-db-upgrade.sh.in +++ b/scripts/pacman-db-upgrade.sh.in @@ -109,18 +109,44 @@ fi # do not let pacman run while we do this touch "$lockfile"
-# pacman-3.4 to 3.5 upgrade - merge depends into desc -if [[ $(find "$dbroot"/local -name depends) ]]; then - msg "$(gettext "Pre-3.5 database format detected - upgrading...")" - for i in "$dbroot"/local/*; do - if [[ -f "$i"/depends ]]; then - cat "$i"/depends >> "$i"/desc - rm "$i"/depends - fi - done - msg "$(gettext "Done.")" +if [[ -f "${dbroot}"/local/.alpm_db_version ]]; then + db_version=($(cat "${dbroot}"/local/.alpm_db_version))
Why is db_version an array?
fi
+if [[ -z "$db_version" ]]; then + # pacman-3.4 to 3.5 upgrade - merge depends into desc + if [[ $(find "$dbroot"/local -name depends) ]]; then + msg "$(gettext "Pre-3.5 database format detected - upgrading...")" + for i in "$dbroot"/local/*; do + if [[ -f "$i"/depends ]]; then + cat "$i"/depends >> "$i"/desc + rm "$i"/depends + fi + done + msg "$(gettext "Done.")" + fi + + # pacman 4.1 to 4.2 upgrade - remove directory symlink support + dirlist=() + + unset GREP_OPTIONS + while IFS= read -r dir; do + dirlist+=("/${dir%/}") + done < <(grep -h '/$' "$dbroot"/local/*/files | sort -u) + + mapfile -t dirlist < <(find "${dirlist[@]}" -maxdepth 0 -type l) + + if [[ ${#dirlist[@]} != 0 ]]; then + msg "$(gettext "Pre-4.2 database format detected - upgrading...")" + for dir in ${dirlist[@]}; do
missing quote, "${dirlist[@]}"
+ realdir=( $(cd $dir; pwd -P) )
missing quote, "$dir"; why is realdir an array?
+ sed -i "s#^${dir#/}/#${realdir#/}/#" "$dbroot"/local/*/files
If it's almost safe to assume paths don't include '\n', it's far less so to assume they don't include '#'. Also $dir is treated as a pattern, not a fixed string here. This should be better: awk -v "dir=${dir#/}/" -v "realdir=${realdir#/}/" ' BEGIN { i = length(dir) + 1 } { if (index($0, dir) == 1) { printf("%s%s\n", realdir, substr($0, i)) } else { print } }' awk doesn't have the convenience of in place file editting though, so we may have to wrap this inside a loop, and store the updated data in a temp file (or just a var?)...
+ done + fi +fi + +echo "9" > "$dbroot"/local/.alpm_db_version + # remove the lock file rm -f "$lockfile"
-- 1.8.3.3
On 08/08/13 04:42, lolilolicon wrote:
+ # pacman 4.1 to 4.2 upgrade - remove directory symlink support
+ dirlist=() + + unset GREP_OPTIONS + while IFS= read -r dir; do + dirlist+=("/${dir%/}") + done < <(grep -h '/$' "$dbroot"/local/*/files | sort -u) + + mapfile -t dirlist < <(find "${dirlist[@]}" -maxdepth 0 -type l) + + if [[ ${#dirlist[@]} != 0 ]]; then + msg "$(gettext "Pre-4.2 database format detected - upgrading...")" + for dir in ${dirlist[@]}; do missing quote, "${dirlist[@]}"
+ realdir=( $(cd $dir; pwd -P) ) missing quote, "$dir"; why is realdir an array?
+ sed -i "s#^${dir#/}/#${realdir#/}/#" "$dbroot"/local/*/files If it's almost safe to assume paths don't include '\n', it's far less so to assume they don't include '#'. Also $dir is treated as a pattern, not a fixed string here. This should be better:
awk -v "dir=${dir#/}/" -v "realdir=${realdir#/}/" ' BEGIN { i = length(dir) + 1 } { if (index($0, dir) == 1) { printf("%s%s\n", realdir, substr($0, i)) } else { print } }'
awk doesn't have the convenience of in place file editting though, so we may have to wrap this inside a loop, and store the updated data in a temp file (or just a var?)...
@Dave: comments here? I believe I got the sed from you - although I am sure it was supposed to be a quick and dirty hack... Allan
On 16/07/13 22:22, Allan McRae wrote:
With removing directory symlink support, we need to make an adjustment to the users local database to adjust any path that relied in that "feature".
As this is not a readily detectable change, as was the case in our last local database adjustment, we need to add something to track the local database version.
The file ".alpm_db_version" is added to the local database root. This file is automatically created whenever the local database directory is absent or empty. This store the current database version as an integer. I have chosen to use the (next) libalpm soname version so it is easy for us to track what versions are needed.
Any comments on these? I had one comment: <toofishes> pacman -Qip should work without a local DB, for instance Lets have a look at pacman-4.1.2: # rm -rf /var/lib/pacman/ # pacman -Qip /tmp/pacman-git-4.1.2_67_g7977aab-1-x86_64.pkg.tar.xz error: failed to initialize alpm library (could not find or read directory) So, we need /var/lib/pacman to initialise the transaction already, so having pacman automatically add the /var/lib/pacman/local directory and a version file is not much of a burden. That means the only change is when you have a local database and it is not the correct version: # pacman -Qip pacman-git-4.1.2_77_g5ee4a65-1-x86_64.pkg.tar.xz error: failed to initialize alpm library (database is incorrect version) error: try running pacman-db-upgrade (the spacing in that output is interesting...) That requires a one-off database upgrade, which will be needed for any pacman operation. Given it blocks ALL pacman operations (which I am fairly sure was the case when we remove "depends" files too), I think this is fine. Allan
On 20/07/13 10:51, Allan McRae wrote:
On 16/07/13 22:22, Allan McRae wrote:
With removing directory symlink support, we need to make an adjustment to the users local database to adjust any path that relied in that "feature".
As this is not a readily detectable change, as was the case in our last local database adjustment, we need to add something to track the local database version.
The file ".alpm_db_version" is added to the local database root. This file is automatically created whenever the local database directory is absent or empty. This store the current database version as an integer. I have chosen to use the (next) libalpm soname version so it is easy for us to track what versions are needed.
Any comments on these?
I had one comment: <toofishes> pacman -Qip should work without a local DB, for instance
Lets have a look at pacman-4.1.2:
# rm -rf /var/lib/pacman/ # pacman -Qip /tmp/pacman-git-4.1.2_67_g7977aab-1-x86_64.pkg.tar.xz error: failed to initialize alpm library (could not find or read directory)
So, we need /var/lib/pacman to initialise the transaction already, so having pacman automatically add the /var/lib/pacman/local directory and a version file is not much of a burden.
That means the only change is when you have a local database and it is not the correct version:
# pacman -Qip pacman-git-4.1.2_77_g5ee4a65-1-x86_64.pkg.tar.xz error: failed to initialize alpm library (database is incorrect version) error: try running pacman-db-upgrade
(the spacing in that output is interesting...)
That requires a one-off database upgrade, which will be needed for any pacman operation. Given it blocks ALL pacman operations (which I am fairly sure was the case when we remove "depends" files too), I think this is fine.
Final request for more comments... Allan
participants (2)
-
Allan McRae
-
lolilolicon