On 10/03/14 at 08:48am, Dave Reisner wrote:

On Thu, Oct 02, 2014 at 04:35:52PM -0400, Andrew Gregory wrote:

...

grep'ing out blank lines and sorting output thoroughly breaks any file lists with %BACKUP% entries which must be separated from the file list by a blank line.

Signed-off-by: Andrew Gregory ---

Anybody who has replaced an owned directory with a symlink and run pacman-db-upgrade has mangled the file list for all packages with backup files. To check for broken file lists run: `grep -xA1 %BACKUP% /var/lib/pacman/local/*/files | grep -B1 %FILES%` Any file lists with output are broken. Broken file lists can be repaired by reinstalling the affected packages with --dbonly. After repairing, users should remove /var/lib/pacman/local/ALPM_DB_VERSION and run the fixed version of pacman-db-upgrade.

Tested this since I was affected -- I don't know what the script did this time, but it *didn't* destroy my local DB. So, +1.

...

scripts/pacman-db-upgrade.sh.in | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/pacman-db-upgrade.sh.in b/scripts/pacman-db-upgrade.sh.in index d2d317b..d7e34bd 100644 --- a/scripts/pacman-db-upgrade.sh.in +++ b/scripts/pacman-db-upgrade.sh.in @@ -211,9 +211,9 @@ if [[ -z "$db_version" ]]; then { if ($0 == olddir) { # replace symlink with its target, including parents - printf("%s", parents) - printf("%s\n", newdir) - } else if ($0 == file) { + if (parents) printf("%s", parents) + if (newdir) printf("%s\n", newdir)

parents and newdir are invariants for the duration of the awk program. Why not skip the whole for loop when [[ -z $parents$newdir ]] ? If you do that, I think your printf here simply becomes:

printf("%s%s\n", parents, newdir)

We can't skip the loop because even if newdir and parents are empty the awk script still has to remove olddir. parents should actually always be safe to print, so this could be reduced to: if(newdir) printf("%s%s\n", parents, newdir) but I think it's easier to read treating parents and newdir independently and similarly.

...

+ } else if (file && $0 == file) { # newdir already existed as a file, skip it } else if (index($0, olddir) == 1) { # update paths that were under olddir @@ -222,7 +222,7 @@ if [[ -z "$db_version" ]]; then # print everything else as-is print } - }' "$f" | grep . | LC_ALL=C sort -u > "$f.tmp" + }' "$f" > "$f.tmp" mv "$f.tmp" "$f" done done -- 2.1.1

On 10/05/14 at 02:25pm, Allan McRae wrote:

On 03/10/14 06:35, Andrew Gregory wrote:

...

grep'ing out blank lines and sorting output thoroughly breaks any file lists with %BACKUP% entries which must be separated from the file list by a blank line.

Signed-off-by: Andrew Gregory ---

Anybody who has replaced an owned directory with a symlink and run pacman-db-upgrade has mangled the file list for all packages with backup files. To check for broken file lists run: `grep -xA1 %BACKUP% /var/lib/pacman/local/*/files | grep -B1 %FILES%` Any file lists with output are broken. Broken file lists can be repaired by reinstalling the affected packages with --dbonly. After repairing, users should remove /var/lib/pacman/local/ALPM_DB_VERSION and run the fixed version of pacman-db-upgrade.

scripts/pacman-db-upgrade.sh.in | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)

diff --git a/scripts/pacman-db-upgrade.sh.in b/scripts/pacman-db-upgrade.sh.in index d2d317b..d7e34bd 100644 --- a/scripts/pacman-db-upgrade.sh.in +++ b/scripts/pacman-db-upgrade.sh.in @@ -211,9 +211,9 @@ if [[ -z "$db_version" ]]; then { if ($0 == olddir) { # replace symlink with its target, including parents - printf("%s", parents) - printf("%s\n", newdir) - } else if ($0 == file) { + if (parents) printf("%s", parents) + if (newdir) printf("%s\n", newdir) + } else if (file && $0 == file) {

Are these changes are unrelated to the issue? Or am I missing something?

These prevent printing blank lines (if the symlink being replaced pointed to the root) since we can't grep them out later.

...

# newdir already existed as a file, skip it } else if (index($0, olddir) == 1) { # update paths that were under olddir @@ -222,7 +222,7 @@ if [[ -z "$db_version" ]]; then # print everything else as-is print } - }' "$f" | grep . | LC_ALL=C sort -u > "$f.tmp" + }' "$f" > "$f.tmp"

So we get duplicate directories in our file list now? I know we sort within pacman, so the order is not an issue.

Can we detect when parents/newdir is printed and avoid printing it again?

That should be doable; parents will have to be split back into individual paths for each file list.

...

mv "$f.tmp" "$f" done done

On 06/10/14 18:16, Andrew Gregory wrote:

grep'ing out blank lines and sorting output thoroughly breaks any file lists with %BACKUP% entries which must be separated from the file list by a blank line. Adds a custom function to ensure that all paths printed are non-empty and unique.

Signed-off-by: Andrew Gregory ---

This is great - pulled to my patchqueue branch Allan

My dbupgrade branch includes a pactest case for this.

scripts/pacman-db-upgrade.sh.in | 29 ++++++++++++++++++++++------- 1 file changed, 22 insertions(+), 7 deletions(-)

diff --git a/scripts/pacman-db-upgrade.sh.in b/scripts/pacman-db-upgrade.sh.in index d2d317b..f9bd101 100644 --- a/scripts/pacman-db-upgrade.sh.in +++ b/scripts/pacman-db-upgrade.sh.in @@ -204,25 +204,40 @@ if [[ -z "$db_version" ]]; then

for f in "$dbroot"/local/*/files; do awk -v "olddir=$olddir" -v "newdir=$newdir" -v "parents=$parents" ' + function print_path(path) { + if (path != "" && !(path in seen)) { + seen[path] = 1 + print path + } + } BEGIN { - i = length(olddir) + 1 + oldlen = length(olddir) + 1 file = substr(newdir, 0, length(newdir) - 1) } { - if ($0 == olddir) { + if ($0 == "") { + # end of section, clear seen paths and print as-is + for ( i in seen ) { + delete seen[i] + } + print + } else if ($0 == olddir) { # replace symlink with its target, including parents - printf("%s", parents) - printf("%s\n", newdir) + split(parents, paths, "\n") + for (i in paths) { + print_path(paths[i]) + } + print_path(newdir) } else if ($0 == file) { # newdir already existed as a file, skip it } else if (index($0, olddir) == 1) { # update paths that were under olddir - printf("%s%s\n", newdir, substr($0, i)) + print_path(newdir substr($0, oldlen)) } else { # print everything else as-is - print + print_path($0) } - }' "$f" | grep . | LC_ALL=C sort -u > "$f.tmp" + }' "$f" > "$f.tmp" mv "$f.tmp" "$f" done done