Documentation on pacman / libalpm package repositories
Hi General, I'm looking for the reference documentation to a pacman / Archlinux http(s) package repository. Specifically, I'm investigating how to host a public repo for the binary artifacts from ceph builds. I'm hoping for a page similar to Debian's on hosting+format [1][2], but any pointers would be much appreciated. Kind regards, and happy new years! Paul Stemmet [1]: https://wiki.debian.org/DebianRepository/Setup [2]: https://wiki.debian.org/DebianRepository/Format -- ~Paul Stemmet email: archlinux.lists at luxolus.com
Hello, Can I please clarify what you are asking. So you would like to host a repo to distribute binaries in which pacman can pull? If so you can check the format of a mirror such as mine: https://archmirror.polarian.dev I do not remember the documentation page but from my memory in /etc/pacman.conf at the bottom you see the list of repositories, you can add your own: [repo name] Server = http(s)://url.tld/ All you got to do is serve binaries using a web server such as pacman. You will need a package list, this I am not too sure about, what I know about it is it is a .tar.gz named as <reponame>.db but is not actually a database (inspected using the file command utility). https://wiki.archlinux.org/title/Pacman Might be useful to check the link above, but I am not sure it is here. I remember finding this referenced in a few of the arch wiki pages, but I am sorry I can't remember which they were (I have read a lot of the arch wiki pages). I will take a look and hopefully reply to this later on with a better reply (if someone else doesn't get there before me). Hope this helps in some way, Polarian
Hello, Just a note, I have looked through the Arch Wiki pages which are related to Arch Mirrors and the pacman package manager, there is no mention of custom repositories, only lists of them. I assumed it is outlined in the man pages for pacman (pacman.conf to be specific). Sorry, Polarian
Hi Polarian, Thanks for the quick response, and yes, I am looking to host binaries that pacman can pull. I'm interested in learning what pacman expects when it comes to file layout -- i.e the developer or reference documentation. Thanks for the pointers -- downloaded the community.db from Archlinux, and am busy poking it now. I'll also checkout your mirror. --- On the wiki pages -- happy to know I'm not just horribly stupid, I am very surprised honestly, as I've found Archlinux's wiki pages impeccable when it comes to information about the distro in the past. Kind Regards, Paul Stemmet On 31/12/2022 22:37, Polarian wrote:
Hello,
Can I please clarify what you are asking.
So you would like to host a repo to distribute binaries in which pacman can pull?
If so you can check the format of a mirror such as mine:
https://archmirror.polarian.dev
I do not remember the documentation page but from my memory in /etc/pacman.conf at the bottom you see the list of repositories, you can add your own:
[repo name] Server = http(s)://url.tld/
All you got to do is serve binaries using a web server such as pacman. You will need a package list, this I am not too sure about, what I know about it is it is a .tar.gz named as <reponame>.db but is not actually a database (inspected using the file command utility).
https://wiki.archlinux.org/title/Pacman
Might be useful to check the link above, but I am not sure it is here. I remember finding this referenced in a few of the arch wiki pages, but I am sorry I can't remember which they were (I have read a lot of the arch wiki pages).
I will take a look and hopefully reply to this later on with a better reply (if someone else doesn't get there before me).
Hope this helps in some way, Polarian
-- ~Paul Stemmet email: archlinux.lists at luxolus.com
Hello, For anyone else who is on this mailing list, do you know if an Arch Wiki page exists on this topic, if so could you link it here. I have checked the DeveloperWiki as well and I can't see any documentation on this, only how to build for official repository, but not explaining the layout. I guess the best option you got so far is to just reverse engineer the repositories unless someone else knows the location of an ArchWiki page covering this topic. Good luck, Polarian
I generally have used instructions from here in the past to create repositories using repo-add which has seems to be worked fine. https://wiki.archlinux.org/title/Pacman/Tips_and_tricks#Custom_local_reposit... An example is here: https://github.com/sutarwadi/arch-hpcstack/blob/75c1db47995d6df4748cd2308144... This seemed to have been enough to generate everything needed to create a signed repo. On Sun, 1 Jan 2023, 00:09 Polarian, <polarian@polarian.dev> wrote:
Hello,
For anyone else who is on this mailing list, do you know if an Arch Wiki page exists on this topic, if so could you link it here.
I have checked the DeveloperWiki as well and I can't see any documentation on this, only how to build for official repository, but not explaining the layout.
I guess the best option you got so far is to just reverse engineer the repositories unless someone else knows the location of an ArchWiki page covering this topic.
Good luck, Polarian
Hello, I knew this was somewhere, I totally forgot it was within the tips and tricks... Maybe it might be a good idea to give this its own dedicated wiki page due to how it is harder to find within the tips and tricks, and how a full guide could be written on creating your own custom repositories. Obviously this will need Arch Wiki maintainers which are interested in writing and maintaining this page, I personally am interested in this, if others are interesting this might be something we can create. Hope this helps you Paul and thank you Jayesh for the help, Polarian
Hi Jayesh, Thanks for that link! I knew it had to be somewhere. Kind Regards, Paul Stemmet On 31/12/2022 23:36, Jayesh Badwaik wrote:
I generally have used instructions from here in the past to create repositories using repo-add which has seems to be worked fine.
https://wiki.archlinux.org/title/Pacman/Tips_and_tricks#Custom_local_reposit...
An example is here: https://github.com/sutarwadi/arch-hpcstack/blob/75c1db47995d6df4748cd2308144...
This seemed to have been enough to generate everything needed to create a signed repo.
On Sun, 1 Jan 2023, 00:09 Polarian, <polarian@polarian.dev> wrote:
Hello,
For anyone else who is on this mailing list, do you know if an Arch Wiki page exists on this topic, if so could you link it here.
I have checked the DeveloperWiki as well and I can't see any documentation on this, only how to build for official repository, but not explaining the layout.
I guess the best option you got so far is to just reverse engineer the repositories unless someone else knows the location of an ArchWiki page covering this topic.
Good luck, Polarian
-- ~Paul Stemmet email: archlinux.lists at luxolus.com
Hey Paul, Interested in expanding this to its own dedicated Arch Wiki page? Thanks, Polarian
Hi Polarian, Yes, I'd be happy to take what I learn and add it as a dedicated page, geared towards the "How to host a third party Archlinux package repo" usecase. I'll be likely exploring this a bunch in the new year, as I plan on splitting up the AUR ceph package into proper libs for the v18 release later in 2023, and the best way to learn is to teach someone else. Kind Regards, Paul Stemmet On 01/01/2023 00:48, Polarian wrote:
Hey Paul,
Interested in expanding this to its own dedicated Arch Wiki page?
Thanks, Polarian
-- ~Paul Stemmet email: archlinux.lists at luxolus.com
On 12/31/22 18:59, Paul Stemmet wrote:
Hi Polarian,
Yes, I'd be happy to take what I learn and add it as a dedicated page, geared towards the "How to host a third party Archlinux package repo" usecase.
I'll be likely exploring this a bunch in the new year, as I plan on splitting up the AUR ceph package into proper libs for the v18 release later in 2023, and the best way to learn is to teach someone else.
Kind Regards, Paul Stemmet
On 01/01/2023 00:48, Polarian wrote:
Hey Paul,
Interested in expanding this to its own dedicated Arch Wiki page?
Thanks, Polarian
If there's already a wiki covering for that, where some people is already familiar, with, why adding a new wiki? I'd suggest improving the existent one, if you find it somehow not fully covering everything needed. -- Javier
Hello, It is in a subsection of another arch wiki page, thus can and should be expanded on if the demand is there. If me and Paul want to expand it I do not see why not? It could be beneficial for others who want to distribute their arch binaries in their own custom repo. Polarian
On 12/31/22 19:17, Polarian wrote:
Hello,
It is in a subsection of another arch wiki page, thus can and should be expanded on if the demand is there.
If me and Paul want to expand it I do not see why not? It could be beneficial for others who want to distribute their arch binaries in their own custom repo.
Polarian
I have no issues against improving/expanding, :) I'd just suggest not scattering the information, separating it into multiple wikis... BTW, I've been using custom repos since I started using Arch, not even remember when was that, :), more than 10 years for sure. That because I maintain some packages of my own, and then because of "aurutils", an AUR helper that takes advantage of something I was already using, local custom repos... It'd be sad to go look over several wikies, instead of just one... Greetings and happy 2023 !! -- Javier
Happy new year you too, Maybe it is time to dedicate a Arch Wiki page to hosting your own repositories to allow more people to be able to host and maintain their own mirrors. If arch linux truly cares about decentralisation, this should be a priority to allow more people to contribute, obviously you would need to inspect and trust a specific repository based on their maintainer reputation (and obviously don't use them if they do not open source their pkgbuilds). We should be able to freely manage our own repositories, its sad that there is not more dedication to allowing self hosting of your own repositories. Thanks for the help, Polarian
On 01/01/2023 04:14, Polarian wrote:
Happy new year you too,
Maybe it is time to dedicate a Arch Wiki page to hosting your own repositories to allow more people to be able to host and maintain their own mirrors. If arch linux truly cares about decentralisation, this should be a priority to allow more people to contribute, obviously you would need to inspect and trust a specific repository based on their maintainer reputation (and obviously don't use them if they do not open source their pkgbuilds).
Plenty of folks who manage to host a third party repository, see the wiki page. [1] [1] https://wiki.archlinux.org/title/Unofficial_user_repositories
Hello, I am aware of this, my recommendation was to add a dedicated page on HOW TO do this, so that it is easier for more people to setup their own repositories to host Arch Packages. Currently there is only a small section within the pacman FAQ explaining how to do it, it would be a lot more useful if it was explained in more depth. Thanks, Polarian
On 2022-12-31 22:29:05 (+0000), Paul Stemmet wrote:
I'm looking for the reference documentation to a pacman / Archlinux http(s) package repository. Specifically, I'm investigating how to host a public repo for the binary artifacts from ceph builds.
I'm hoping for a page similar to Debian's on hosting+format [1][2], but any pointers would be much appreciated.
Kind regards, and happy new years! Paul Stemmet
[1]: https://wiki.debian.org/DebianRepository/Setup [2]: https://wiki.debian.org/DebianRepository/Format
-- ~Paul Stemmet email: archlinux.lists at luxolus.com
Although in practice not in use for our repositories yet, you may want to have a look at the repod documentation: https://repod.archlinux.page Best, David -- https://sleepmap.de
participants (6)
-
David Runge
-
Javier
-
Jayesh Badwaik
-
Jelle van der Waa
-
Paul Stemmet
-
Polarian