[pacman-dev] Versioned packages
Here is my attempt to solve seven years old infamous problem: https://bugs.archlinux.org/task/16702
Patch won't solve problem out of the box, a small changes in kernel PKGBUILD will be required, but only concerning install part.
Idea behind patch is pretty simple: 1) Configure list of packages and number of old versions pacman should try to preserve. 2) When upgading to new version, keep old in place, if it has no file conflicts with new one, and mark it as `archived`, remove oldest `archived` version instead.
Most of time pacman treats `archived` packages as if they aren't installed. For now it won't check package conflicts and dependencies, only file conflicts with newer versions. It's only an outline of full solution, proof of concept to illustrate the idea.
I'd like to hear opinion of community whether this problem should be solved at all, or is it more like a feature of ArchLinux, and if it should, whether such approach suits ArchLinux's philosophy.
On 10/09/16 08:41, Sergey Petrenko via pacman-dev wrote:
Here is my attempt to solve seven years old infamous problem: https://bugs.archlinux.org/task/16702
Patch won't solve problem out of the box, a small changes in kernel PKGBUILD will be required, but only concerning install part.
Idea behind patch is pretty simple:
- Configure list of packages and number of old versions pacman should try
to preserve. 2) When upgading to new version, keep old in place, if it has no file conflicts with new one, and mark it as `archived`, remove oldest `archived` version instead.
Most of time pacman treats `archived` packages as if they aren't installed. For now it won't check package conflicts and dependencies, only file conflicts with newer versions. It's only an outline of full solution, proof of concept to illustrate the idea.
I'd like to hear opinion of community whether this problem should be solved at all, or is it more like a feature of ArchLinux, and if it should, whether such approach suits ArchLinux's philosophy.
How is this better than having a package file sitting in the cache?
The "kernel problem" in Arch is not because it is not possible to have multiple kernel packages available. Other distributions provide endless amounts of kernels (e.g. Manjaro).
I don't see anything that needs done on the package manager end for this.
Allan
Should I spam kernel package maintainers then, or maybe someone will resolve bug as wontfix?
Суббота, 10 сентября 2016, 0:58 +03:00 от Allan McRae allan@archlinux.org:
On 10/09/16 08:41, Sergey Petrenko via pacman-dev wrote:
Here is my attempt to solve seven years old infamous problem: https://bugs.archlinux.org/task/16702
Patch won't solve problem out of the box, a small changes in kernel PKGBUILD will be required, but only concerning install part.
Idea behind patch is pretty simple:
- Configure list of packages and number of old versions pacman should try
to preserve. 2) When upgading to new version, keep old in place, if it has no file conflicts with new one, and mark it as `archived`, remove oldest `archived` version instead.
Most of time pacman treats `archived` packages as if they aren't installed. For now it won't check package conflicts and dependencies, only file conflicts with newer versions. It's only an outline of full solution, proof of concept to illustrate the idea.
I'd like to hear opinion of community whether this problem should be solved at all, or is it more like a feature of ArchLinux, and if it should, whether such approach suits ArchLinux's philosophy.
How is this better than having a package file sitting in the cache?
The "kernel problem" in Arch is not because it is not possible to have multiple kernel packages available. Other distributions provide endless amounts of kernels (e.g. Manjaro).
I don't see anything that needs done on the package manager end for this.
Allan
On 09/12/16 at 09:47am, Sergey Petrenko via pacman-dev wrote:
Should I spam kernel package maintainers then, or maybe someone will resolve bug as wontfix?
Not sure why they need to be spammed, you can easily build linux47 as a package and install it separate from the normal linux package. But I guess you want to automatically retain your current installed linux pkg when you upgrade to a newer version?
Суббота, 10 сентября 2016, 0:58 +03:00 от Allan McRae allan@archlinux.org:
On 10/09/16 08:41, Sergey Petrenko via pacman-dev wrote:
Here is my attempt to solve seven years old infamous problem: https://bugs.archlinux.org/task/16702
Patch won't solve problem out of the box, a small changes in kernel PKGBUILD will be required, but only concerning install part.
Idea behind patch is pretty simple:
- Configure list of packages and number of old versions pacman should try
to preserve. 2) When upgading to new version, keep old in place, if it has no file conflicts with new one, and mark it as `archived`, remove oldest `archived` version instead.
Most of time pacman treats `archived` packages as if they aren't installed. For now it won't check package conflicts and dependencies, only file conflicts with newer versions. It's only an outline of full solution, proof of concept to illustrate the idea.
I'd like to hear opinion of community whether this problem should be solved at all, or is it more like a feature of ArchLinux, and if it should, whether such approach suits ArchLinux's philosophy.
How is this better than having a package file sitting in the cache?
The "kernel problem" in Arch is not because it is not possible to have multiple kernel packages available. Other distributions provide endless amounts of kernels (e.g. Manjaro).
I don't see anything that needs done on the package manager end for this.
Allan
-- With wish of constant improvement and unstoppable creativity. Sergey Petrenko
Well, of course! I can also build kernel outside package management, or write a hook to backup kernels, but I'd like to see solution that would not require such dire and time consuming measures, and, ideally, would not require actions from me at all.
Понедельник, 12 сентября 2016, 10:22 +03:00 от Jelle van der Waa jelle@vdwaa.nl:
On 09/12/16 at 09:47am, Sergey Petrenko via pacman-dev wrote:
Should I spam kernel package maintainers then, or maybe someone will resolve bug as wontfix?
Not sure why they need to be spammed, you can easily build linux47 as a package and install it separate from the normal linux package. But I guess you want to automatically retain your current installed linux pkg when you upgrade to a newer version?
Суббота, 10 сентября 2016, 0:58 +03:00 от Allan McRae < allan@archlinux.org >:
On 10/09/16 08:41, Sergey Petrenko via pacman-dev wrote:
Here is my attempt to solve seven years old infamous problem: https://bugs.archlinux.org/task/16702
Patch won't solve problem out of the box, a small changes in kernel PKGBUILD will be required, but only concerning install part.
Idea behind patch is pretty simple:
- Configure list of packages and number of old versions pacman should try
to preserve. 2) When upgading to new version, keep old in place, if it has no file conflicts with new one, and mark it as `archived`, remove oldest `archived` version instead.
Most of time pacman treats `archived` packages as if they aren't installed. For now it won't check package conflicts and dependencies, only file conflicts with newer versions. It's only an outline of full solution, proof of concept to illustrate the idea.
I'd like to hear opinion of community whether this problem should be solved at all, or is it more like a feature of ArchLinux, and if it should, whether such approach suits ArchLinux's philosophy.
How is this better than having a package file sitting in the cache?
The "kernel problem" in Arch is not because it is not possible to have multiple kernel packages available. Other distributions provide endless amounts of kernels (e.g. Manjaro).
I don't see anything that needs done on the package manager end for this.
Allan
-- With wish of constant improvement and unstoppable creativity. Sergey Petrenko
-- Jelle van der Waa
On 12/09, Sergey Petrenko via pacman-dev wrote:
Well, of course! I can also build kernel outside package management, or write a hook to backup kernels, but I'd like to see solution that would not require such dire and time consuming measures, and, ideally, would not require actions from me at all.
Then maybe Arch is not for you.
On 12/09/16 22:59, Johannes Löthberg wrote:
On 12/09, Sergey Petrenko via pacman-dev wrote:
Well, of course! I can also build kernel outside package management, or write a hook to backup kernels, but I'd like to see solution that would not require such dire and time consuming measures, and, ideally, would not require actions from me at all.
Then maybe Arch is not for you.
This discussion has nothing to do with Arch. It is about whether this feature needs implemented in pacman.
Allan
On 09/12/2016 06:51 PM, Allan McRae wrote:
This discussion has nothing to do with Arch. It is about whether this feature needs implemented in pacman.
Which of course it doesn't. All we need is for the kernel maintainer to start packaging a dummy linux package that depends on linux-$pkgver packages.
As originally recommended by the bugtracker item which spawned this odd thread.
...
I am not quite sure what is currently blocking such a resolution, but that is already supposedly being discussed on the bugtracker.
Meanwhile let us listen to people proposing convoluted workarounds that evade the issue entirely and are far less likely to actually be accepted...
On Mon, 2016-09-12 at 21:33 -0400, Eli Schwartz wrote:
On 09/12/2016 06:51 PM, Allan McRae wrote:
This discussion has nothing to do with Arch. It is about whether this feature needs implemented in pacman.
Which of course it doesn't. All we need is for the kernel maintainer to start packaging a dummy linux package that depends on linux-$pkgver packages.
As originally recommended by the bugtracker item which spawned this odd thread.
...
I am not quite sure what is currently blocking such a resolution, but that is already supposedly being discussed on the bugtracker.
Meanwhile let us listen to people proposing convoluted workarounds that evade the issue entirely and are far less likely to actually be accepted...
What about adding a new option to pacman's '-D, --database' operator which would enable users to rename packages and add a corresponding provide field. This would empower everyone to decide on their own which packages they want in different versions and which they don't want. Pacman would handle file conflicts as usual without introducing a new kind of packages. Plus it would avoid discussions about which package should be a dummy package and which shouldn't be.
Best Regards, Gordian Edenhofer
On 09/19/2016 11:49 AM, Gordian Edenhofer wrote:
What about adding a new option to pacman's '-D, --database' operator which would enable users to rename packages and add a corresponding provide field. This would empower everyone to decide on their own which packages they want in different versions and which they don't want. Pacman would handle file conflicts as usual without introducing a new kind of packages. Plus it would avoid discussions about which package should be a dummy package and which shouldn't be.
Was that meant to be some sort of a joke? Because even Debian is not that insane, AFAIK.
On Tue, 2016-09-20 at 21:41 -0400, Eli Schwartz wrote:
On 09/19/2016 11:49 AM, Gordian Edenhofer wrote:
What about adding a new option to pacman's '-D, --database' operator which would enable users to rename packages and add a corresponding provide field. This would empower everyone to decide on their own which packages they want in different versions and which they don't want. Pacman would handle file conflicts as usual without introducing a new kind of packages. Plus it would avoid discussions about which package should be a dummy package and which shouldn't be.
Was that meant to be some sort of a joke? Because even Debian is not that insane, AFAIK.
You bar for insanity is whether the debian folks do it? Please explain yourself why do you think this is the wrong approach for the problem. My reasoning would be that packaging a dummy package would result in the same 'renamed' package on the system but would require the package maintainer to take action. Btw. my Arch system is rock solid and I am not especially interested in dummy packages on my system but for those who need it it can be a huge difference. Therefore it makes sense to give those specific user base the ability to do so on their own. It wouldn't even be a dangerous operation since by adding a provide field one should be unable to brick the system. However I admit that it would require some additions to pacman which is obviously an argument against this but the way packages are treated would stay the same. Currently I don't see a reason why one should change the way of packaging just to circumvent the underlying problem.
On 09/21/2016 05:56 AM, Gordian Edenhofer wrote:
You bar for insanity is whether the debian folks do it? Please explain yourself why do you think this is the wrong approach for the problem. My reasoning would be that packaging a dummy package would result in the same 'renamed' package on the system but would require the package maintainer to take action.
Well, I can usually count on them to be over-engineered.
But in all seriousness, then. :) Building in to the package manager, a capacity for retroactively modifying the definition of a specific package, seems to me to be a clear case of over-engineering. And just a bad idea in principle.
If you want to do that, it is because you are already doing *something* wrong. Because, you should not have to change your mind about what these particular files actually are, and start calling them something else on a whim. Particularly as it disappears from the repositories, stops being trusted as signed by a central authority (the TUs), cannot be reinstalled...
Btw. my Arch system is rock solid and I am not especially interested in dummy packages on my system but for those who need it it can be a huge difference. Therefore it makes sense to give those specific user base the ability to do so on their own. It wouldn't even be a dangerous operation since by adding a provide field one should be unable to brick the system. However I admit that it would require some additions to pacman which is obviously an argument against this but the way packages are treated would stay the same. Currently I don't see a reason why one should change the way of packaging just to circumvent the underlying problem.
I am a bit confused as to what this all means...
Do you prefer dummy packages which pull in versioned kernels and depend on changing the packaging (for that specific package)? Or do you prefer additions to pacman while "treating the packages the same"?
...
I think we may have a drastically different idea of what constitutes "circumventing the underlying problem". To be specific, we both seem to think the other is the one circumventing the underlying problem.
On Wed, 2016-09-21 at 23:28 -0400, Eli Schwartz wrote:
On 09/21/2016 05:56 AM, Gordian Edenhofer wrote:
You bar for insanity is whether the debian folks do it? Please explain yourself why do you think this is the wrong approach for the problem. My reasoning would be that packaging a dummy package would result in the same 'renamed' package on the system but would require the package maintainer to take action.
Well, I can usually count on them to be over-engineered.
But in all seriousness, then. :) Building in to the package manager, a capacity for retroactively modifying the definition of a specific package, seems to me to be a clear case of over-engineering. And just a bad idea in principle.
If you want to do that, it is because you are already doing *something* wrong.
I wouldn't assume that the user has done anything wrong, rather something with the system is wrong if one feels the need to 'backup' packages.
Because, you should not have to change your mind about what these particular files actually are, and start calling them something else on
Since one is intending to backup a package one should be highly aware of the purpose of the package.
a whim. Particularly as it disappears from the repositories, stops being trusted as signed by a central authority (the TUs), cannot be reinstalled...
The provide field makes sure that it is still treated the same. But to allow backups for all packages, the user has to live with them not being signed-off.
Btw. my Arch system is rock solid and I am not especially interested in dummy packages on my system but for those who need it it can be a huge difference. Therefore it makes sense to give those specific user base the ability to do so on their own. It wouldn't even be a dangerous operation since by adding a provide field one should be unable to brick the system. However I admit that it would require some additions to pacman which is obviously an argument against this but the way packages are treated would stay the same. Currently I don't see a reason why one should change the way of packaging just to circumvent the underlying problem.
I am a bit confused as to what this all means...
Do you prefer dummy packages which pull in versioned kernels and depend on changing the packaging (for that specific package)? Or do you prefer additions to pacman while "treating the packages the same"?
I dislike dummy packages and therefore would keep the current way of creating e.g. the linux package. If using dummy packages the main question becomes where to draw the line. You are currently referring to linux as "for that specific package" but what if users need different versions for another package... The system of everyone else would become more and more cluttered with dummy packages.
I think we may have a drastically different idea of what constitutes "circumventing the underlying problem". To be specific, we both seem to think the other is the one circumventing the underlying problem.
That's true :D
On 09/22/2016 03:53 AM, Gordian Edenhofer wrote:
I wouldn't assume that the user has done anything wrong, rather something with the system is wrong if one feels the need to 'backup' packages.
Awesome, we must be in agreement, because I don't think people should feel the need to backup package*s* either.
Since one is intending to backup a package one should be highly aware of the purpose of the package.
What does that have to do with anything???
The provide field makes sure that it is still treated the same. But to allow backups for all packages, the user has to live with them not being signed-off.
Okay. What about reinstalling this mysterious ghost-like package? And when, precisely, did you decide that "backups for all packages" was a desirable feature, rather than a highly kludgy workaround for the edge case of the linux kernel?
I dislike dummy packages and therefore would keep the current way of creating e.g. the linux package. If using dummy packages the main question becomes where to draw the line. You are currently referring to linux as "for that specific package" but what if users need different versions for another package... The system of everyone else would become more and more cluttered with dummy packages.
To add to my previous objections: Arch Linux doesn't believe in running old software, and the linux is a unique exception whereby we don't want to delete a currently-running kernel, the solution to which is not "let's modify pacman".
Maybe the source of this confusion is, you should read the bugtracker to see why this thread was initially proposed, and then decide if backing up package*s* is a good solution. Rather than presupposing it is a wanted feature. (feature creep) https://bugs.archlinux.org/task/16702
I have already said that this feature has no place in pacman. Please take it off list if you want to continue discussing it.
Allan
Unfortunately, some changes are required to testing framework to handle multiple versions of package. New tests should make expected behavior of `archived` packages pretty clear.
--- test/pacman/pmdb.py | 51 +++++++++++++++++++++++++++++++--------- test/pacman/pmpkg.py | 2 ++ test/pacman/pmrule.py | 17 ++++++++++++++ test/pacman/tests/TESTS | 5 ++++ test/pacman/tests/archived001.py | 22 +++++++++++++++++ test/pacman/tests/archived002.py | 31 ++++++++++++++++++++++++ test/pacman/tests/archived003.py | 37 +++++++++++++++++++++++++++++ test/pacman/tests/archived004.py | 31 ++++++++++++++++++++++++ test/pacman/tests/archived005.py | 29 +++++++++++++++++++++++ test/pacman/tests/archived006.py | 39 ++++++++++++++++++++++++++++++ 10 files changed, 253 insertions(+), 11 deletions(-) create mode 100644 test/pacman/tests/archived001.py create mode 100644 test/pacman/tests/archived002.py create mode 100644 test/pacman/tests/archived003.py create mode 100644 test/pacman/tests/archived004.py create mode 100644 test/pacman/tests/archived005.py create mode 100644 test/pacman/tests/archived006.py
diff --git a/test/pacman/pmdb.py b/test/pacman/pmdb.py index 1f20506..db2f270 100644 --- a/test/pacman/pmdb.py +++ b/test/pacman/pmdb.py @@ -60,6 +60,7 @@ def __init__(self, treename, root): self.is_local = True self.read_dircache = None self.read_pkgcache = {} + self.archived = {} else: self.dbdir = None self.dbfile = os.path.join(root, util.PM_SYNCDBPATH, treename + ".db") @@ -79,35 +80,53 @@ def getpkg(self, name): if name == pkg.name: return pkg
- def db_read(self, name): + def db_read(self, name, archived=False): if not self.dbdir or not os.path.isdir(self.dbdir): return None
- dbentry = None + if not archived and name in self.read_pkgcache: + return self.read_pkgcache[name] + if archived and name in self.archived: + return self.archived[name] + if self.read_dircache is None: self.read_dircache = os.listdir(self.dbdir) + + candidates = [] for entry in self.read_dircache: if entry == "ALPM_DB_VERSION": continue [pkgname, pkgver, pkgrel] = entry.rsplit("-", 2) if pkgname == name: - dbentry = entry - break - if dbentry is None: + candidates.append(entry) + if len(candidates) == 0: return None
- if pkgname in self.read_pkgcache: - return self.read_pkgcache[pkgname] + for candidate in candidates: + self.pkg_load(candidate) + + if name not in self.archived: + self.archived[name] = [] + + if not archived: + if name in self.read_pkgcache: + return self.read_pkgcache[name] + else: + return self.archived[name] + + return None
+ def pkg_load(self, dbentry): + + [pkgname, pkgver, pkgrel] = dbentry.rsplit("-", 2) pkg = pmpkg.pmpkg(pkgname, pkgver + "-" + pkgrel) - self.read_pkgcache[pkgname] = pkg
path = os.path.join(self.dbdir, dbentry) # desc filename = os.path.join(path, "desc") if not os.path.isfile(filename): tap.bail("invalid db entry found (desc missing) for pkg " + pkgname) - return None + return fd = open(filename, "r") while 1: line = fd.readline() @@ -136,6 +155,12 @@ def db_read(self, name): except ValueError: pkg.reason = -1 raise + elif line == "%ARCHIVED%": + try: + pkg.archived = int(fd.readline().strip("\n")) + except ValueError: + pkg.archived = -1 + raise elif line == "%SIZE%" or line == "%CSIZE%": try: pkg.size = int(fd.readline().strip("\n")) @@ -162,7 +187,7 @@ def db_read(self, name): filename = os.path.join(path, "files") if not os.path.isfile(filename): tap.bail("invalid db entry found (files missing) for pkg " + pkgname) - return None + return fd = open(filename, "r") while 1: line = fd.readline() @@ -181,7 +206,10 @@ def db_read(self, name): # install filename = os.path.join(path, "install")
- return pkg + if pkg.archived: + self.archived.setdefault(pkgname, []).append(pkg) + else: + self.read_pkgcache[pkgname] = pkg
# # db_write is used to add both 'local' and 'sync' db entries @@ -207,6 +235,7 @@ def db_write(self, pkg): make_section(data, "INSTALLDATE", pkg.installdate) make_section(data, "SIZE", pkg.size) make_section(data, "REASON", pkg.reason) + make_section(data, "ARCHIVED", pkg.archived) else: make_section(data, "FILENAME", pkg.filename()) make_section(data, "REPLACES", pkg.replaces) diff --git a/test/pacman/pmpkg.py b/test/pacman/pmpkg.py index 8a88a35..5b7dd34 100644 --- a/test/pacman/pmpkg.py +++ b/test/pacman/pmpkg.py @@ -43,6 +43,7 @@ def __init__(self, name, version = "1.0-1"): self.csize = 0 self.isize = 0 self.reason = 0 + self.archived = 0 self.md5sum = "" # sync only self.pgpsig = "" # sync only self.replaces = [] @@ -71,6 +72,7 @@ def __str__(self): s.append("url: %s" % self.url) s.append("files: %s" % " ".join(self.files)) s.append("reason: %d" % self.reason) + s.append("archived: %d" % self.archived) return "\n".join(s)
def fullname(self): diff --git a/test/pacman/pmrule.py b/test/pacman/pmrule.py index 61eded0..2f178a6 100644 --- a/test/pacman/pmrule.py +++ b/test/pacman/pmrule.py @@ -111,6 +111,23 @@ def check(self, test): else: tap.diag("PKG rule '%s' not found" % case) success = -1 + elif kind == "ARCHIVED-PKG": + localdb = test.db["local"] + pkgs = localdb.db_read(key, True) + if not pkgs: + success = 0 + else: + if case == "EXIST": + success = 1 + elif case == "VERSION": + if all(pkg.version != value for pkg in pkgs): + success = 0 + elif case == "FILES": + if all(value not in pkg.files for pkg in pkgs): + success = 0 + else: + tap.diag("ARCHIVED-PKG rule '%s' not found" % case) + success = -1 elif kind == "FILE": filename = os.path.join(test.root, key) if case == "EXIST": diff --git a/test/pacman/tests/TESTS b/test/pacman/tests/TESTS index bd5a0b6..f70123b 100644 --- a/test/pacman/tests/TESTS +++ b/test/pacman/tests/TESTS @@ -1,3 +1,8 @@ +TESTS += test/pacman/tests/archived001.py +TESTS += test/pacman/tests/archived002.py +TESTS += test/pacman/tests/archived003.py +TESTS += test/pacman/tests/archived004.py +TESTS += test/pacman/tests/archived005.py TESTS += test/pacman/tests/backup001.py TESTS += test/pacman/tests/clean001.py TESTS += test/pacman/tests/clean002.py diff --git a/test/pacman/tests/archived001.py b/test/pacman/tests/archived001.py new file mode 100644 index 0000000..19d554b --- /dev/null +++ b/test/pacman/tests/archived001.py @@ -0,0 +1,22 @@ +self.description = "Upgrade a package, archive previous version" + +self.option["MaxArchived"] = [1] +self.option["ArchivePkg"] = ["dummy"] + +lp = pmpkg("dummy") +lp.files = ["bin/dummy1", + "usr/man/man1/dummy1.1"] +self.addpkg2db("local", lp) + +p = pmpkg("dummy", "2.0-1") +p.files = ["bin/dummy2", + "usr/man/man1/dummy2.1"] +self.addpkg(p) + +self.args = "-U %s" % p.filename() + +self.addrule("PACMAN_RETCODE=0") +self.addrule("PKG_VERSION=dummy|2.0-1") +self.addrule("ARCHIVED-PKG_VERSION=dummy|1.0-1") +for f in lp.files + p.files: + self.addrule("FILE_EXIST=%s" % f) diff --git a/test/pacman/tests/archived002.py b/test/pacman/tests/archived002.py new file mode 100644 index 0000000..58136f4 --- /dev/null +++ b/test/pacman/tests/archived002.py @@ -0,0 +1,31 @@ +self.description = "Upgrade a package, archive previous version, remove old archived" + +self.option["MaxArchived"] = [1] +self.option["ArchivePkg"] = ["dummy"] + +ap = pmpkg("dummy") +ap.files = ["bin/dummy1", + "usr/man/man1/dummy1.1"] +ap.archived = 1 +self.addpkg2db("local", ap) + +lp = pmpkg("dummy", "2.0-1") +lp.files = ["bin/dummy2", + "usr/man/man1/dummy2.1"] +self.addpkg2db("local", lp) + +p = pmpkg("dummy", "3.0-1") +p.files = ["bin/dummy3", + "usr/man/man1/dummy3.1"] +self.addpkg(p) + +self.args = "-U %s" % p.filename() + +self.addrule("PACMAN_RETCODE=0") +self.addrule("PKG_VERSION=dummy|3.0-1") +self.addrule("ARCHIVED-PKG_VERSION=dummy|2.0-1") +self.addrule("!ARCHIVED-PKG_VERSION=dummy|1.0-1") +for f in lp.files + p.files: + self.addrule("FILE_EXIST=%s" % f) +for f in ap.files: + self.addrule("!FILE_EXIST=%s" % f) diff --git a/test/pacman/tests/archived003.py b/test/pacman/tests/archived003.py new file mode 100644 index 0000000..544eb0b --- /dev/null +++ b/test/pacman/tests/archived003.py @@ -0,0 +1,37 @@ +self.description = "Downgrade a package, remove newer archived" + +self.option["MaxArchived"] = [2] +self.option["ArchivePkg"] = ["dummy"] + +ap1 = pmpkg("dummy") +ap1.files = ["bin/dummy1", + "usr/man/man1/dummy1.1"] +ap1.archived = 1 +self.addpkg2db("local", ap1) + +ap2 = pmpkg("dummy", "2.0-1") +ap2.files = ["bin/dummy2", + "usr/man/man1/dummy2.1"] +ap2.archived = 1 +self.addpkg2db("local", ap2) + +lp = pmpkg("dummy", "3.0-1") +lp.files = ["bin/dummy3", + "usr/man/man1/dummy3.1"] +self.addpkg2db("local", lp) + +p = pmpkg("dummy", "1.5-1") +p.files = ["bin/dummy1.5", + "usr/man/man1/dummy1.5.1"] +self.addpkg(p) + +self.args = "-U %s" % p.filename() + +self.addrule("PACMAN_RETCODE=0") +self.addrule("PKG_VERSION=dummy|1.5-1") +self.addrule("ARCHIVED-PKG_VERSION=dummy|1.0-1") +self.addrule("!ARCHIVED-PKG_VERSION=dummy|2.0-1") +for f in lp.files + ap2.files: + self.addrule("!FILE_EXIST=%s" % f) +for f in p.files + ap1.files: + self.addrule("FILE_EXIST=%s" % f) diff --git a/test/pacman/tests/archived004.py b/test/pacman/tests/archived004.py new file mode 100644 index 0000000..5bf4431 --- /dev/null +++ b/test/pacman/tests/archived004.py @@ -0,0 +1,31 @@ +self.description = "Upgrade a archivable package with file conflicts" + +self.option["MaxArchived"] = [1] +self.option["ArchivePkg"] = ["dummy"] + +ap = pmpkg("dummy") +ap.files = ["bin/dummy1", + "usr/man/man1/dummy1.1"] +ap.archived = 1 +self.addpkg2db("local", ap) + +lp = pmpkg("dummy", "2.0-1") +lp.files = ["bin/dummy2", + "usr/man/man1/dummy2.1"] +self.addpkg2db("local", lp) + +p = pmpkg("dummy", "2.0-2") +p.files = ["bin/dummy2", + "usr/man/man1/dummy2.2.1"] +self.addpkg(p) + +self.args = "-U %s" % p.filename() + +self.addrule("PACMAN_RETCODE=0") +self.addrule("PKG_VERSION=dummy|2.0-2") +self.addrule("ARCHIVED-PKG_VERSION=dummy|1.0-1") +self.addrule("!ARCHIVED-PKG_VERSION=dummy|2.0-1") +for f in ap.files + p.files: + self.addrule("FILE_EXIST=%s" % f) +for f in set(lp.files) - set(p.files): + self.addrule("!FILE_EXIST=%s" % f) diff --git a/test/pacman/tests/archived005.py b/test/pacman/tests/archived005.py new file mode 100644 index 0000000..ad5a289 --- /dev/null +++ b/test/pacman/tests/archived005.py @@ -0,0 +1,29 @@ +self.description = "Remove a package with archive" + +self.option["MaxArchived"] = [2] +self.option["ArchivePkg"] = ["dummy"] + +ap1 = pmpkg("dummy") +ap1.files = ["bin/dummy1", + "usr/man/man1/dummy1.1"] +ap1.archived = 1 +self.addpkg2db("local", ap1) + +ap2 = pmpkg("dummy", "2.0-1") +ap2.files = ["bin/dummy2", + "usr/man/man1/dummy2.1"] +ap2.archived = 1 +self.addpkg2db("local", ap2) + +lp = pmpkg("dummy", "3.0-1") +lp.files = ["bin/dummy3", + "usr/man/man1/dummy3.1"] +self.addpkg2db("local", lp) + +self.args = "-R %s" % lp.name + +self.addrule("PACMAN_RETCODE=0") +self.addrule("!PKG_EXIST=dummy") +self.addrule("!ARCHIVED-PKG_EXIST=dummy") +for f in lp.files + ap2.files + ap1.files: + self.addrule("!FILE_EXIST=%s" % f) diff --git a/test/pacman/tests/archived006.py b/test/pacman/tests/archived006.py new file mode 100644 index 0000000..e06e38a --- /dev/null +++ b/test/pacman/tests/archived006.py @@ -0,0 +1,39 @@ +self.description = "Reinstall a package with archive" + +self.option["MaxArchived"] = [2] +self.option["ArchivePkg"] = ["dummy"] + +ap1 = pmpkg("dummy") +ap1.files = ["bin/dummy1", + "usr/man/man1/dummy1.1"] +ap1.archived = 1 +self.addpkg2db("local", ap1) + +ap2 = pmpkg("dummy", "2.0-1") +ap2.files = ["bin/dummy2", + "usr/man/man1/dummy2.1"] +ap2.archived = 1 +self.addpkg2db("local", ap2) + +lp = pmpkg("dummy", "3.0-1") +lp.files = ["bin/dummy3", + "usr/man/man1/dummy3.1"] +self.addpkg2db("local", lp) + +p = pmpkg("dummy", "3.0-1") +p.files = ["bin/dummy", + "usr/man/man1/dummy3.1"] +self.addpkg(p) + +self.args = "-U %s" % p.filename() + +self.addrule("PACMAN_RETCODE=0") +self.addrule("PKG_VERSION=dummy|3.0-1") +self.addrule("ARCHIVED-PKG_VERSION=dummy|1.0-1") +self.addrule("ARCHIVED-PKG_VERSION=dummy|2.0-1") +for f in set(lp.files) - set(p.files): + self.addrule("!FILE_EXIST=%s" % f) +for f in p.files: + self.addrule("FILE_MODIFIED=%s" % f) +for f in ap1.files + ap2.files + p.files: + self.addrule("FILE_EXIST=%s" % f)
I had gave a big thought to the question, who should decide whether to try to archive package.
At first glance it looks like maintainer's responsibility. She/he/it creates a package without file conflicts intending possibility for multiple versions to be installed on a system. Yet this means boolean flag on packages in sync repository. Such flag will look really really ugly (at least to me).
Other option is no option at all. Pacman always checks file conflicts between old and new versions and can try to archive any packages that has none. I guess such approach wouldn't change a bit neither in speed of pacman operations, nor in file system of users with current state of sync databases. It looks like every package has at least one common file between versions. Yet at some point such packages may appear and why would user want to keep multiple versions of package neither she/he/it, nor maintainer intended to be `archived`?
And thus solution I have ended with - new config option. `MaxArchived` specifies window of versions to `archive`, and `ArchivePkg` - patterns of packages' names.
In this patch `archived` packages are handled only in update and remove operations. Of course additional changes are required to display `archived` packages in queries, possibly package-level conflicts checking for `archived` packages is also required.
--- lib/libalpm/add.c | 177 +++++++++++++++++++++++++++++++++++++++++++++++-- lib/libalpm/alpm.h | 28 +++++++- lib/libalpm/be_local.c | 62 ++++++++++++++--- lib/libalpm/conflict.c | 25 ++++++- lib/libalpm/db.c | 22 ++++++ lib/libalpm/db.h | 3 + lib/libalpm/filelist.c | 54 +++++++++++++++ lib/libalpm/filelist.h | 5 ++ lib/libalpm/handle.c | 38 +++++++++++ lib/libalpm/handle.h | 2 + lib/libalpm/package.c | 13 ++++ lib/libalpm/package.h | 2 + lib/libalpm/pkghash.c | 48 ++++++++++++++ lib/libalpm/pkghash.h | 8 +++ lib/libalpm/remove.c | 16 +++++ src/pacman/callback.c | 21 ++++++ src/pacman/conf.c | 20 ++++++ src/pacman/conf.h | 2 + 18 files changed, 529 insertions(+), 17 deletions(-)
diff --git a/lib/libalpm/add.c b/lib/libalpm/add.c index d132e52..eb035d7 100644 --- a/lib/libalpm/add.c +++ b/lib/libalpm/add.c @@ -46,6 +46,7 @@ #include "db.h" #include "remove.h" #include "handle.h" +#include "filelist.h"
/** Add a package to the transaction. */ int SYMEXPORT alpm_add_pkg(alpm_handle_t *handle, alpm_pkg_t *pkg) @@ -393,6 +394,173 @@ static int extract_single_file(alpm_handle_t *handle, struct archive *archive, return errors; }
+static int should_archive_oldpkg(alpm_pkg_t *oldpkg, alpm_package_operation_t operation) +{ + return oldpkg->reason == ALPM_PKG_REASON_EXPLICIT && + oldpkg->handle->max_archived > 0 && + operation != ALPM_PACKAGE_REINSTALL && + alpm_list_find(oldpkg->handle->archivepkg, oldpkg->name, _alpm_fnmatch) != NULL; +} + +static int should_remove_archivable_oldpkg(alpm_pkg_t *oldpkg, alpm_pkg_t *newpkg, + alpm_package_operation_t operation, int *files_overlap) +{ + if (operation == ALPM_PACKAGE_DOWNGRADE) { + return 1; + } + *files_overlap = _alpm_filelist_overlap(&oldpkg->files, &newpkg->files); + return *files_overlap; +} + +static alpm_errno_t ask_to_remove_archived_pkgs(alpm_handle_t *handle, + alpm_pkg_t *newpkg, alpm_list_t *archived) +{ + alpm_errno_t ret = (alpm_errno_t)0; + + if (archived) { + alpm_question_remove_from_archive_t question = { + .type = ALPM_QUESTION_REMOVE_FROM_ARCHIVE, + .remove = 0, + .pkgs = archived + }; + + QUESTION(handle, &question); + if(question.remove) { + alpm_list_t *itr; + for (itr = archived; itr; itr = alpm_list_next(archived)) { + alpm_pkg_t *archived_pkg = itr->data; + + _alpm_log(handle, ALPM_LOG_DEBUG, _("removing archived %s-%s\n"), + archived_pkg->name, archived_pkg->version); + + if(_alpm_remove_single_package(handle, archived_pkg, newpkg, 0, 0) == -1) { + ret = ALPM_ERR_TRANS_ABORT; + goto cleanup; + } + } + } else { + ret = ALPM_ERR_TRANS_ABORT; + goto cleanup; + } + } + +cleanup: + alpm_list_free(archived); + return ret; +} + +static alpm_list_t *get_old_and_conficting_archived_pkgs(alpm_handle_t *handle, + alpm_pkg_t *newpkg, int oldpkg_files_overlap) +{ + alpm_list_t *archived = _alpm_db_get_archived_pkgs(handle->db_local, newpkg->name); + alpm_list_t *itr = NULL; + alpm_list_t *archived_descending = alpm_list_reverse(archived); + int max_archived_allowed = handle->max_archived; + + alpm_list_free(archived); + archived = NULL; + + if (!oldpkg_files_overlap) { + /* We need place for one new archived package. */ + max_archived_allowed -= 1; + } + + for (itr = archived_descending; itr && max_archived_allowed > 0; itr = alpm_list_next(itr)) { + alpm_list_t *tmp_itr; + alpm_pkg_t *archived_pkg = itr->data; + + if (_alpm_filelist_overlap(&archived_pkg->files, &newpkg->files)) { + continue; + } + + tmp_itr = alpm_list_next(itr); + archived_descending = alpm_list_remove_item(archived_descending, itr); + max_archived_allowed -= 1; + free(itr); + itr = tmp_itr; + } + + return archived_descending; +} + +static alpm_errno_t archive_pkg(alpm_handle_t *handle, alpm_pkg_t *oldpkg) +{ + /* Removing package copy from pkgcache */ + alpm_pkg_t *set_me_free; + _alpm_pkghash_remove(_alpm_db_get_pkgcache_hash(handle->db_local), oldpkg, &set_me_free); + _alpm_pkg_free(set_me_free); + + _alpm_log(handle, ALPM_LOG_DEBUG, _("archiving %s-%s\n"), + oldpkg->name, oldpkg->version); + + oldpkg->archived = 1; + /* Writing new `archived` flag to db. */ + if(_alpm_local_db_write(handle->db_local, oldpkg, INFRQ_DESC)) { + _alpm_log(handle, ALPM_LOG_ERROR, _("could not update database entry %s-%s\n"), + oldpkg->name, oldpkg->version); + alpm_logaction(handle, ALPM_CALLER_PREFIX, + "error: could not update database entry %s-%s\n", + oldpkg->name, oldpkg->version); + return ALPM_ERR_DB_WRITE; + } + + _alpm_pkghash_add_sorted(_alpm_db_get_pkgcache_hash(handle->db_local), oldpkg); + + return (alpm_errno_t)0; +} + +static alpm_errno_t archive_or_remove_oldpkg(alpm_handle_t *handle, alpm_pkg_t *oldpkg, + alpm_pkg_t *newpkg, alpm_package_operation_t operation) +{ + alpm_db_t *db = handle->db_local; + int oldpkg_files_overlap = 0; + int try_archive; + int can_archive = 0; + + try_archive = should_archive_oldpkg(oldpkg, operation); + if (try_archive) { + can_archive = !should_remove_archivable_oldpkg(oldpkg, newpkg, operation, + &oldpkg_files_overlap); + } + if (!can_archive) { + if (oldpkg_files_overlap) { + _alpm_log(handle, ALPM_LOG_WARNING, _("Can not archive entry %s-%s, " + "files overlap with new package\n"), + oldpkg->name, oldpkg->version); + } + /* set up fake remove transaction */ + if(_alpm_remove_single_package(handle, oldpkg, newpkg, 0, 0) == -1) { + return ALPM_ERR_TRANS_ABORT; + } + } + if (!try_archive) { + return (alpm_errno_t)0; + } + + if (operation == ALPM_PACKAGE_DOWNGRADE) { + return ask_to_remove_archived_pkgs(handle, newpkg, + _alpm_db_get_archived_pkgs_newer(db, newpkg->name, newpkg->version)); + } else if (operation == ALPM_PACKAGE_UPGRADE) { + + alpm_list_t *archived = get_old_and_conficting_archived_pkgs(handle, newpkg, + oldpkg_files_overlap); + alpm_errno_t result = ask_to_remove_archived_pkgs(handle, newpkg, archived); + if (result != (alpm_errno_t)0) { + return result; + } + + if (can_archive) { + result = archive_pkg(handle, oldpkg); + if (result != (alpm_errno_t)0) { + return result; + } + newpkg->oldpkg = NULL; + } + } + + return (alpm_errno_t)0; +} + static int commit_single_pkg(alpm_handle_t *handle, alpm_pkg_t *newpkg, size_t pkg_current, size_t pkg_count) { @@ -457,10 +625,11 @@ static int commit_single_pkg(alpm_handle_t *handle, alpm_pkg_t *newpkg, newpkg->reason = ALPM_PKG_REASON_EXPLICIT; }
- if(oldpkg) { - /* set up fake remove transaction */ - if(_alpm_remove_single_package(handle, oldpkg, newpkg, 0, 0) == -1) { - handle->pm_errno = ALPM_ERR_TRANS_ABORT; + if(oldpkg) { + alpm_errno_t pm_errno = archive_or_remove_oldpkg(handle, oldpkg, newpkg, + event.operation); + if(pm_errno != (alpm_errno_t)0) { + handle->pm_errno = pm_errno; ret = -1; goto cleanup; } diff --git a/lib/libalpm/alpm.h b/lib/libalpm/alpm.h index 168d71b..8d3df03 100644 --- a/lib/libalpm/alpm.h +++ b/lib/libalpm/alpm.h @@ -612,7 +612,8 @@ typedef enum _alpm_question_type_t { ALPM_QUESTION_CORRUPTED_PKG = (1 << 3), ALPM_QUESTION_REMOVE_PKGS = (1 << 4), ALPM_QUESTION_SELECT_PROVIDER = (1 << 5), - ALPM_QUESTION_IMPORT_KEY = (1 << 6) + ALPM_QUESTION_IMPORT_KEY = (1 << 6), + ALPM_QUESTION_REMOVE_FROM_ARCHIVE = (1 << 7) } alpm_question_type_t;
typedef struct _alpm_question_any_t { @@ -693,6 +694,15 @@ typedef struct _alpm_question_import_key_t { alpm_pgpkey_t *key; } alpm_question_import_key_t;
+typedef struct _alpm_question_remove_from_archive_t { + /** Type of question. */ + alpm_question_type_t type; + /** Answer: whether or not to remove archived versions. */ + int remove; + /** List of archived alpm_pkg_t* which are to be removed. */ + alpm_list_t *pkgs; +} alpm_question_remove_from_archive_t; + /** * Questions. * This is an union passed to the callback, that allows the frontend to know @@ -709,6 +719,7 @@ typedef union _alpm_question_t { alpm_question_remove_pkgs_t remove_pkgs; alpm_question_select_provider_t select_provider; alpm_question_import_key_t import_key; + alpm_question_remove_from_archive_t remove_from_archive; } alpm_question_t;
/** Question callback */ @@ -869,6 +880,18 @@ int alpm_option_remove_noextract(alpm_handle_t *handle, const char *path); int alpm_option_match_noextract(alpm_handle_t *handle, const char *path); /** @} */
+/** @name Accessors to the list of archived packages. + * These functions modify the list of packages several old + * versions of which should be preserved on filesystem when possible. + * @{ + */ +alpm_list_t *alpm_option_get_archivepkgs(alpm_handle_t *handle); +int alpm_option_add_archivepkg(alpm_handle_t *handle, const char *archivepkg); +int alpm_option_set_archivepkgs(alpm_handle_t *handle, alpm_list_t *archivepkgs); +int alpm_option_remove_archivepkg(alpm_handle_t *handle, const char *archivepkg); +/** @} */ + + /** @name Accessors to the list of ignored packages. * These functions modify the list of packages that * should be ignored by a sysupgrade. @@ -910,6 +933,9 @@ int alpm_option_set_arch(alpm_handle_t *handle, const char *arch); double alpm_option_get_deltaratio(alpm_handle_t *handle); int alpm_option_set_deltaratio(alpm_handle_t *handle, double ratio);
+int alpm_option_get_maxarchived(alpm_handle_t *handle); +int alpm_option_set_maxarchived(alpm_handle_t *handle, int max_archived); + int alpm_option_get_checkspace(alpm_handle_t *handle); int alpm_option_set_checkspace(alpm_handle_t *handle, int checkspace);
diff --git a/lib/libalpm/be_local.c b/lib/libalpm/be_local.c index cc2d8ba..e0d7761 100644 --- a/lib/libalpm/be_local.c +++ b/lib/libalpm/be_local.c @@ -556,7 +556,7 @@ static int local_db_populate(alpm_db_t *db) while((ent = readdir(dbdir)) != NULL) { const char *name = ent->d_name;
- alpm_pkg_t *pkg; + alpm_pkg_t *pkg, *dbpkg;
if(strcmp(name, ".") == 0 || strcmp(name, "..") == 0) { continue; @@ -579,18 +579,32 @@ static int local_db_populate(alpm_db_t *db) continue; }
- /* duplicated database entries are not allowed */ - if(_alpm_pkghash_find(db->pkgcache, pkg->name)) { - _alpm_log(db->handle, ALPM_LOG_ERROR, _("duplicated database entry '%s'\n"), pkg->name); - _alpm_pkg_free(pkg); - continue; - } - pkg->origin = ALPM_PKG_FROM_LOCALDB; pkg->origin_data.db = db; pkg->ops = &local_pkg_ops; pkg->handle = db->handle;
+ dbpkg = _alpm_pkghash_find(db->pkgcache, pkg->name); + if (dbpkg != NULL) { + /* Reading 'DESC' to distinquish which one is archived. + * Actually, could be done with vercmp, but such approach + * won't detect inconsistances in local db. + */ + local_db_read(dbpkg, INFRQ_DESC); + local_db_read(pkg, INFRQ_DESC); + if (dbpkg->archived) { + alpm_pkg_t tmp = *dbpkg; + *dbpkg = *pkg; + *pkg = tmp; + } else if (!pkg->archived) { + /* duplicated database entries without `archived` flag are not allowed */ + _alpm_log(db->handle, ALPM_LOG_ERROR, _("duplicated database entry '%s'\n"), + pkg->name); + _alpm_pkg_free(pkg); + continue; + } + } + /* explicitly read with only 'BASE' data, accessors will handle the rest */ if(local_db_read(pkg, INFRQ_BASE) == -1) { _alpm_log(db->handle, ALPM_LOG_ERROR, _("corrupted database entry '%s'\n"), name); @@ -607,7 +621,10 @@ static int local_db_populate(alpm_db_t *db)
closedir(dbdir); if(count > 0) { - db->pkgcache->list = alpm_list_msort(db->pkgcache->list, (size_t)count, _alpm_pkg_cmp); + size_t archived_count = alpm_list_count(db->pkgcache->archive); + + db->pkgcache->list = alpm_list_msort(db->pkgcache->list, (size_t)db->pkgcache->entries, _alpm_pkg_cmp); + db->pkgcache->archive = alpm_list_msort(db->pkgcache->archive, archived_count, _alpm_pkg_cmp); } _alpm_log(db->handle, ALPM_LOG_DEBUG, "added %d packages to package cache for db '%s'\n", count, db->treename); @@ -689,6 +706,26 @@ static int local_db_read(alpm_pkg_t *info, alpm_dbinfrq_t inforeq) /* clear out 'line', to be certain - and to make valgrind happy */ memset(line, 0, sizeof(line));
+ /* BASE */ + if (inforeq & INFRQ_BASE) { + char *path = _alpm_local_db_pkgpath(db, info, "archived"); + if(!path) { + _alpm_log(db->handle, ALPM_LOG_ERROR, _("could not open file %s: %s\n"), path, strerror(errno)); + free(path); + goto error; + } + if ((fp = fopen(path, "r")) != NULL){ + if (fscanf(fp, "%d", &info->archived) == EOF) { + _alpm_log(db->handle, ALPM_LOG_ERROR, _("could not read archived flag from file %s: %s\n"), path, strerror(errno)); + free(path); + goto error; + } + fclose(fp); + fp = NULL; + } + free(path); + } + /* DESC */ if(inforeq & INFRQ_DESC && !(info->infolevel & INFRQ_DESC)) { char *path = _alpm_local_db_pkgpath(db, info, "desc"); @@ -741,6 +778,9 @@ static int local_db_read(alpm_pkg_t *info, alpm_dbinfrq_t inforeq) } else if(strcmp(line, "%REASON%") == 0) { READ_NEXT(); info->reason = (alpm_pkgreason_t)atoi(line); + } else if(strcmp(line, "%ARCHIVED%") == 0) { + READ_NEXT(); + info->archived = atoi(line); } else if(strcmp(line, "%VALIDATION%") == 0) { alpm_list_t *i, *v = NULL; READ_AND_STORE_ALL(v); @@ -978,6 +1018,10 @@ int _alpm_local_db_write(alpm_db_t *db, alpm_pkg_t *info, alpm_dbinfrq_t inforeq fprintf(fp, "%%REASON%%\n" "%u\n\n", info->reason); } + if(info->archived) { + fprintf(fp, "%%ARCHIVED%%\n" + "%u\n\n", info->archived); + } if(info->groups) { fputs("%GROUPS%\n", fp); for(lp = info->groups; lp; lp = lp->next) { diff --git a/lib/libalpm/conflict.c b/lib/libalpm/conflict.c index 092b019..0db59ad 100644 --- a/lib/libalpm/conflict.c +++ b/lib/libalpm/conflict.c @@ -385,6 +385,25 @@ static alpm_list_t *alpm_db_find_file_owners(alpm_db_t* db, const char *path) return owners; }
+static alpm_list_t *_alpm_filelist_difference_on_all_versions( + alpm_filelist_t *newpkg_filelist, alpm_db_t *db, alpm_pkg_t *oldpkg) +{ + alpm_list_t *archived, *itr; + alpm_list_t *tmpfiles = _alpm_filelist_difference(newpkg_filelist, + alpm_pkg_get_files(oldpkg)); + + archived = _alpm_db_get_archived_pkgs(db, oldpkg->name); + for (itr = archived; itr; itr = alpm_list_next(itr)) { + alpm_pkg_t *archived_pkg = itr->data; + + tmpfiles = _alpm_list_filelist_difference(tmpfiles, + alpm_pkg_get_files(archived_pkg)); + } + alpm_list_free(archived); + + return tmpfiles; +} + /** * @brief Find file conflicts that may occur during the transaction. * @@ -482,9 +501,9 @@ alpm_list_t *_alpm_db_find_fileconflicts(alpm_handle_t *handle, * that the former list needs to be freed while the latter list should NOT * be freed. */ if(dbpkg) { - /* older ver of package currently installed */ - tmpfiles = _alpm_filelist_difference(alpm_pkg_get_files(p1), - alpm_pkg_get_files(dbpkg)); + /* older verions (including archived) of package currently installed */ + tmpfiles = _alpm_filelist_difference_on_all_versions(alpm_pkg_get_files(p1), + handle->db_local, dbpkg); } else { /* no version of package currently installed */ alpm_filelist_t *fl = alpm_pkg_get_files(p1); diff --git a/lib/libalpm/db.c b/lib/libalpm/db.c index f70f83c..85606a1 100644 --- a/lib/libalpm/db.c +++ b/lib/libalpm/db.c @@ -527,6 +527,8 @@ void _alpm_db_free_pkgcache(alpm_db_t *db) if(db->pkgcache) { alpm_list_free_inner(db->pkgcache->list, (alpm_list_fn_free)_alpm_pkg_free); + alpm_list_free_inner(db->pkgcache->archive, + (alpm_list_fn_free)_alpm_pkg_free); _alpm_pkghash_free(db->pkgcache); } db->status &= ~DB_STATUS_PKGCACHE; @@ -636,6 +638,26 @@ alpm_pkg_t *_alpm_db_get_pkgfromcache(alpm_db_t *db, const char *target) return _alpm_pkghash_find(pkgcache, target); }
+alpm_list_t *_alpm_db_get_archived_pkgs(alpm_db_t *db, const char *name) +{ + return _alpm_db_get_archived_pkgs_newer(db, name, NULL); +} + +alpm_list_t *_alpm_db_get_archived_pkgs_newer(alpm_db_t *db, const char *name, + const char *version) +{ + if (db == NULL) { + return NULL; + } + + alpm_pkghash_t *pkgcache = _alpm_db_get_pkgcache_hash(db); + if(!pkgcache) { + return NULL; + } + + return _alpm_pkghash_get_archived_pkgs_newer(pkgcache, name, version); +} + /* Returns a new group cache from db. */ static int load_grpcache(alpm_db_t *db) diff --git a/lib/libalpm/db.h b/lib/libalpm/db.h index 05ef43e..eef2255 100644 --- a/lib/libalpm/db.h +++ b/lib/libalpm/db.h @@ -103,6 +103,9 @@ int _alpm_db_remove_pkgfromcache(alpm_db_t *db, alpm_pkg_t *pkg); alpm_pkghash_t *_alpm_db_get_pkgcache_hash(alpm_db_t *db); alpm_list_t *_alpm_db_get_pkgcache(alpm_db_t *db); alpm_pkg_t *_alpm_db_get_pkgfromcache(alpm_db_t *db, const char *target); +alpm_list_t *_alpm_db_get_archived_pkgs(alpm_db_t *db, const char *name); +alpm_list_t *_alpm_db_get_archived_pkgs_newer(alpm_db_t *db, const char *name, + const char *version); /* groups */ alpm_list_t *_alpm_db_get_groupcache(alpm_db_t *db); alpm_group_t *_alpm_db_get_groupfromcache(alpm_db_t *db, const char *target); diff --git a/lib/libalpm/filelist.c b/lib/libalpm/filelist.c index 5783373..fd74e75 100644 --- a/lib/libalpm/filelist.c +++ b/lib/libalpm/filelist.c @@ -61,6 +61,35 @@ alpm_list_t *_alpm_filelist_difference(alpm_filelist_t *filesA, return ret; }
+alpm_list_t *_alpm_list_filelist_difference(alpm_list_t *tmpfiles, + alpm_filelist_t *filesB) +{ + alpm_list_t *itrA = tmpfiles; + size_t ctrB = 0; + + while(itrA && ctrB < filesB->count) { + char *strA = itrA->data; + char *strB = filesB->files[ctrB].name; + + int cmp = strcmp(strA, strB); + if(cmp < 0) { + /* item only in filesA, qualifies as a difference */ + itrA = alpm_list_next(itrA); + } else if(cmp > 0) { + ctrB++; + } else { + alpm_list_t *tmp_itr = alpm_list_next(itrA); + tmpfiles = alpm_list_remove_item(tmpfiles, itrA); + free(itrA); + itrA = tmp_itr; + + ctrB++; + } + } + + return tmpfiles; +} + static int _alpm_filelist_pathcmp(const char *p1, const char *p2) { while(*p1 && *p1 == *p2) { @@ -109,6 +138,31 @@ alpm_list_t *_alpm_filelist_intersection(alpm_filelist_t *filesA, return ret; }
+int _alpm_filelist_overlap(alpm_filelist_t *filesA, alpm_filelist_t *filesB) +{ + size_t ctrA = 0, ctrB = 0; + alpm_file_t *arrA = filesA->files, *arrB = filesB->files; + + while(ctrA < filesA->count && ctrB < filesB->count) { + const char *strA = arrA[ctrA].name, *strB = arrB[ctrB].name; + int cmp = _alpm_filelist_pathcmp(strA, strB); + if(cmp < 0) { + ctrA++; + } else if(cmp > 0) { + ctrB++; + } else { + /* when not directories, item in both qualifies as an intersect */ + if(strA[strlen(strA) - 1] != '/' || strB[strlen(strB) - 1] != '/') { + return 1; + } + ctrA++; + ctrB++; + } + } + + return 0; +} + /* Helper function for comparing files list entries */ int _alpm_files_cmp(const void *f1, const void *f2) diff --git a/lib/libalpm/filelist.h b/lib/libalpm/filelist.h index 5560ea0..f1a7a5e 100644 --- a/lib/libalpm/filelist.h +++ b/lib/libalpm/filelist.h @@ -24,9 +24,14 @@ alpm_list_t *_alpm_filelist_difference(alpm_filelist_t *filesA, alpm_filelist_t *filesB);
+alpm_list_t *_alpm_list_filelist_difference(alpm_list_t *tmpfiles, + alpm_filelist_t *filesB); + alpm_list_t *_alpm_filelist_intersection(alpm_filelist_t *filesA, alpm_filelist_t *filesB);
+int _alpm_filelist_overlap(alpm_filelist_t *filesA, alpm_filelist_t *filesB); + int _alpm_files_cmp(const void *f1, const void *f2);
#endif /* _ALPM_FILELIST_H */ diff --git a/lib/libalpm/handle.c b/lib/libalpm/handle.c index e9439a0..7e944f2 100644 --- a/lib/libalpm/handle.c +++ b/lib/libalpm/handle.c @@ -92,6 +92,7 @@ void _alpm_handle_free(alpm_handle_t *handle) FREELIST(handle->noextract); FREELIST(handle->ignorepkg); FREELIST(handle->ignoregroup); + FREELIST(handle->archivepkg);
alpm_list_free_inner(handle->assumeinstalled, (alpm_list_fn_free)alpm_dep_free); alpm_list_free(handle->assumeinstalled); @@ -271,6 +272,12 @@ alpm_list_t SYMEXPORT *alpm_option_get_noextracts(alpm_handle_t *handle) return handle->noextract; }
+alpm_list_t SYMEXPORT *alpm_option_get_archivepkgs(alpm_handle_t *handle) +{ + CHECK_HANDLE(handle, return NULL); + return handle->archivepkg; +} + alpm_list_t SYMEXPORT *alpm_option_get_ignorepkgs(alpm_handle_t *handle) { CHECK_HANDLE(handle, return NULL); @@ -301,6 +308,12 @@ double SYMEXPORT alpm_option_get_deltaratio(alpm_handle_t *handle) return handle->deltaratio; }
+int SYMEXPORT alpm_option_get_maxarchived(alpm_handle_t *handle) +{ + CHECK_HANDLE(handle, return -1); + return handle->max_archived; +} + int SYMEXPORT alpm_option_get_checkspace(alpm_handle_t *handle) { CHECK_HANDLE(handle, return -1); @@ -627,6 +640,21 @@ int SYMEXPORT alpm_option_match_noextract(alpm_handle_t *handle, const char *pat return _alpm_fnmatch_patterns(handle->noextract, path); }
+int SYMEXPORT alpm_option_add_archivepkg(alpm_handle_t *handle, const char *archivepkg) +{ + return _alpm_option_strlist_add(handle, &(handle->archivepkg), archivepkg); +} + +int SYMEXPORT alpm_option_set_archivepkgs(alpm_handle_t *handle, alpm_list_t *archivepkg) +{ + return _alpm_option_strlist_set(handle, &(handle->archivepkg), archivepkg); +} + +int SYMEXPORT alpm_option_remove_archivepkg(alpm_handle_t *handle, const char *archivepkg) +{ + return _alpm_option_strlist_rem(handle, &(handle->archivepkg), archivepkg); +} + int SYMEXPORT alpm_option_add_ignorepkg(alpm_handle_t *handle, const char *pkg) { return _alpm_option_strlist_add(handle, &(handle->ignorepkg), pkg); @@ -742,6 +770,16 @@ int SYMEXPORT alpm_option_set_deltaratio(alpm_handle_t *handle, double ratio) return 0; }
+int SYMEXPORT alpm_option_set_maxarchived(alpm_handle_t *handle, int max_archived) +{ + CHECK_HANDLE(handle, return -1); + if(max_archived < 0) { + RET_ERR(handle, ALPM_ERR_WRONG_ARGS, -1); + } + handle->max_archived = max_archived; + return 0; +} + alpm_db_t SYMEXPORT *alpm_get_localdb(alpm_handle_t *handle) { CHECK_HANDLE(handle, return NULL); diff --git a/lib/libalpm/handle.h b/lib/libalpm/handle.h index a1d0f9a..31aeb02 100644 --- a/lib/libalpm/handle.h +++ b/lib/libalpm/handle.h @@ -90,10 +90,12 @@ struct __alpm_handle_t { alpm_list_t *ignorepkg; /* List of packages to ignore */ alpm_list_t *ignoregroup; /* List of groups to ignore */ alpm_list_t *assumeinstalled; /* List of virtual packages used to satisfy dependencies */ + alpm_list_t *archivepkg; /* List of packages to archive */
/* options */ char *arch; /* Architecture of packages we should allow */ double deltaratio; /* Download deltas if possible; a ratio value */ + int max_archived; /* Maximum number of package versions to archive */ int usesyslog; /* Use syslog instead of logfile? */ /* TODO move to frontend */ int checkspace; /* Check disk space before installing */ char *dbext; /* Sync DB extension */ diff --git a/lib/libalpm/package.c b/lib/libalpm/package.c index f08df8b..c3cc9c8 100644 --- a/lib/libalpm/package.c +++ b/lib/libalpm/package.c @@ -591,6 +591,7 @@ int _alpm_pkg_dup(alpm_pkg_t *pkg, alpm_pkg_t **new_ptr) newpkg->scriptlet = pkg->scriptlet; newpkg->reason = pkg->reason; newpkg->validation = pkg->validation; + newpkg->archived = pkg->archived;
newpkg->licenses = alpm_list_strdup(pkg->licenses); newpkg->replaces = list_depdup(pkg->replaces); @@ -727,6 +728,18 @@ int _alpm_pkg_cmp(const void *p1, const void *p2) return strcmp(pkg1->name, pkg2->name); }
+int _alpm_pkg_version_cmp(const void *p1, const void *p2) +{ + const alpm_pkg_t *pkg1 = p1; + const alpm_pkg_t *pkg2 = p2; + int res = strcmp(pkg1->name, pkg2->name); + if (res == 0) { + return alpm_pkg_vercmp(pkg1->version, pkg2->version); + } else { + return res; + } +} + /* Test for existence of a package in a alpm_list_t* * of alpm_pkg_t* */ diff --git a/lib/libalpm/package.h b/lib/libalpm/package.h index 31add9a..26a0b54 100644 --- a/lib/libalpm/package.h +++ b/lib/libalpm/package.h @@ -134,6 +134,7 @@ struct __alpm_pkg_t { alpm_pkgvalidation_t validation; alpm_pkgfrom_t origin; alpm_pkgreason_t reason; + int archived; int scriptlet; };
@@ -151,6 +152,7 @@ alpm_pkg_t *_alpm_pkg_load_internal(alpm_handle_t *handle, const char *pkgfile, int full);
int _alpm_pkg_cmp(const void *p1, const void *p2); +int _alpm_pkg_version_cmp(const void *p1, const void *p2); int _alpm_pkg_compare_versions(alpm_pkg_t *local_pkg, alpm_pkg_t *pkg);
#endif /* _ALPM_PACKAGE_H */ diff --git a/lib/libalpm/pkghash.c b/lib/libalpm/pkghash.c index 968218e..aaa298a 100644 --- a/lib/libalpm/pkghash.c +++ b/lib/libalpm/pkghash.c @@ -166,6 +166,15 @@ static alpm_pkghash_t *pkghash_add_pkg(alpm_pkghash_t *hash, alpm_pkg_t *pkg, return hash; }
+ if (pkg->archived) { + if (!sorted) { + hash->archive = alpm_list_add(hash->archive, pkg); + } else { + hash->archive = alpm_list_add_sorted(hash->archive, pkg, _alpm_pkg_version_cmp); + } + return hash; + } + if(hash->entries >= hash->limit) { hash = rehash(hash); } @@ -251,6 +260,12 @@ alpm_pkghash_t *_alpm_pkghash_remove(alpm_pkghash_t *hash, alpm_pkg_t *pkg, return hash; }
+ if (pkg->archived) { + hash->archive = alpm_list_remove(hash->archive, pkg, _alpm_pkg_version_cmp, + (void**)data); + return hash; + } + position = pkg->name_hash % hash->buckets; while((i = hash->hash_table[position]) != NULL) { alpm_pkg_t *info = i->data; @@ -312,6 +327,7 @@ void _alpm_pkghash_free(alpm_pkghash_t *hash) } free(hash->hash_table); } + alpm_list_free(hash->archive); free(hash); }
@@ -345,4 +361,36 @@ alpm_pkg_t *_alpm_pkghash_find(alpm_pkghash_t *hash, const char *name) return NULL; }
+alpm_list_t *_alpm_pkghash_get_archived_pkgs(alpm_pkghash_t *hash, const char *name) +{ + return _alpm_pkghash_get_archived_pkgs_newer(hash, name, NULL); +} + +alpm_list_t *_alpm_pkghash_get_archived_pkgs_newer(alpm_pkghash_t *hash, + const char *name, const char *version) +{ + alpm_list_t *result = NULL; + alpm_list_t *itr; + + if(name == NULL || hash == NULL) { + return NULL; + } + + for (itr = hash->archive; itr; itr = alpm_list_next(itr)) { + alpm_pkg_t *archived = itr->data; + int name_cmp = strcmp(archived->name, name); + + if (name_cmp > 0) { + break; + } + + if (name_cmp == 0 && (version == NULL || + alpm_pkg_vercmp(archived->version, version) >= 0)) { + result = alpm_list_add(result, archived); + } + } + + return result; +} + /* vim: set noet: */ diff --git a/lib/libalpm/pkghash.h b/lib/libalpm/pkghash.h index c843a43..119edd1 100644 --- a/lib/libalpm/pkghash.h +++ b/lib/libalpm/pkghash.h @@ -37,6 +37,10 @@ struct __alpm_pkghash_t { alpm_list_t **hash_table; /** head node of the hash table data in normal list format */ alpm_list_t *list; + + /** sorted by name and version list of archived packages */ + alpm_list_t *archive; + /** number of buckets in hash table */ unsigned int buckets; /** number of entries in hash table */ @@ -57,4 +61,8 @@ void _alpm_pkghash_free(alpm_pkghash_t *hash);
alpm_pkg_t *_alpm_pkghash_find(alpm_pkghash_t *hash, const char *name);
+alpm_list_t *_alpm_pkghash_get_archived_pkgs(alpm_pkghash_t *hash, const char *name); +alpm_list_t *_alpm_pkghash_get_archived_pkgs_newer(alpm_pkghash_t *hash, + const char *name, const char *version); + #endif /* _ALPM_PKGHASH_H */ diff --git a/lib/libalpm/remove.c b/lib/libalpm/remove.c index 45f7c2f..cda5e54 100644 --- a/lib/libalpm/remove.c +++ b/lib/libalpm/remove.c @@ -59,6 +59,7 @@ int SYMEXPORT alpm_remove_pkg(alpm_handle_t *handle, alpm_pkg_t *pkg) const char *pkgname; alpm_trans_t *trans; alpm_pkg_t *copy; + alpm_list_t *itr, *archived;
/* Sanity checks */ CHECK_HANDLE(handle, return -1); @@ -83,6 +84,21 @@ int SYMEXPORT alpm_remove_pkg(alpm_handle_t *handle, alpm_pkg_t *pkg) return -1; } trans->remove = alpm_list_add(trans->remove, copy); + + archived = _alpm_db_get_archived_pkgs(handle->db_local, pkgname); + for (itr = archived; itr; itr = alpm_list_next(itr)) { + alpm_pkg_t *archived_pkg = itr->data; + + _alpm_log(handle, ALPM_LOG_DEBUG, "adding archived package %s-%s to the transaction remove list\n", + pkgname, archived_pkg->version); + if(_alpm_pkg_dup(archived_pkg, ©) == -1) { + alpm_list_free(archived); + return -1; + } + trans->remove = alpm_list_add(trans->remove, copy); + } + alpm_list_free(archived); + return 0; }
diff --git a/src/pacman/callback.c b/src/pacman/callback.c index 7f72b84..35a2c00 100644 --- a/src/pacman/callback.c +++ b/src/pacman/callback.c @@ -487,6 +487,27 @@ void cb_question(alpm_question_t *question) } } break; + case ALPM_QUESTION_REMOVE_FROM_ARCHIVE: + { + alpm_question_remove_from_archive_t *q = &question->remove_from_archive; + alpm_list_t *versionlist = NULL, *i; + size_t count = 0; + for(i = q->pkgs; i; i = i->next) { + versionlist = alpm_list_add(versionlist, + (char *)alpm_pkg_get_version(i->data)); + count++; + } + colon_printf(_n( + "The following archived version of package %s have to be removed:\n", + "The following archived versions of package %s have to be removed:\n", + count), alpm_pkg_get_name(q->pkgs->data)); + list_display(" ", versionlist, getcols()); + printf("\n"); + q->remove = yesno(_("Proceed?")); + alpm_list_free(versionlist); + } + break; + } if(config->noask) { if(config->ask & question->type) { diff --git a/src/pacman/conf.c b/src/pacman/conf.c index 25de7af..1f2ffaa 100644 --- a/src/pacman/conf.c +++ b/src/pacman/conf.c @@ -150,6 +150,7 @@ int config_free(config_t *oldconfig) free(oldconfig->gpgdir); FREELIST(oldconfig->hookdirs); FREELIST(oldconfig->cachedirs); + FREELIST(oldconfig->archivepkg); free(oldconfig->xfercommand); free(oldconfig->print_format); free(oldconfig->arch); @@ -524,6 +525,23 @@ static int _parse_options(const char *key, char *value, if(!config->arch) { config_set_arch(value); } + } else if(strcmp(key, "ArchivePkg") == 0) { + setrepeatingoption(value, "ArchivePkg", &(config->archivepkg)); + } else if(strcmp(key, "MaxArchived") == 0) { + long max_archived; + char *endptr; + + max_archived = strtol(value, &endptr, 10); + + if(*endptr != '\0' || max_archived < 0 || max_archived > USHRT_MAX) { + pm_printf(ALPM_LOG_ERROR, + _("config file %s, line %d: invalid value for '%s' : '%li'\n"), + file, linenum, "MaxArchived", max_archived); + return 1; + } + + config->max_archived = (unsigned short)max_archived; + pm_printf(ALPM_LOG_DEBUG, "config: maxarchived: %li\n", max_archived); } else if(strcmp(key, "UseDelta") == 0) { double ratio; char *endptr; @@ -810,11 +828,13 @@ static int setup_libalpm(void) alpm_option_set_checkspace(handle, config->checkspace); alpm_option_set_usesyslog(handle, config->usesyslog); alpm_option_set_deltaratio(handle, config->deltaratio); + alpm_option_set_maxarchived(handle, config->max_archived);
alpm_option_set_ignorepkgs(handle, config->ignorepkg); alpm_option_set_ignoregroups(handle, config->ignoregrp); alpm_option_set_noupgrades(handle, config->noupgrade); alpm_option_set_noextracts(handle, config->noextract); + alpm_option_set_archivepkgs(handle, config->archivepkg);
for(i = config->assumeinstalled; i; i = i->next) { char *entry = i->data; diff --git a/src/pacman/conf.h b/src/pacman/conf.h index 2aba8bf..19676c0 100644 --- a/src/pacman/conf.h +++ b/src/pacman/conf.h @@ -56,6 +56,7 @@ typedef struct __config_t { unsigned short usesyslog; unsigned short color; double deltaratio; + unsigned short max_archived; char *arch; char *print_format; /* unfortunately, we have to keep track of paths both here and in the library @@ -120,6 +121,7 @@ typedef struct __config_t { alpm_list_t *assumeinstalled; alpm_list_t *noupgrade; alpm_list_t *noextract; + alpm_list_t *archivepkg; char *xfercommand;
/* our connection to libalpm */
participants (6)
-
Allan McRae
-
Eli Schwartz
-
Gordian Edenhofer
-
Jelle van der Waa
-
Johannes Löthberg
-
Sergey Petrenko