Give me cake, or give me debug symbols. - Some comedian, probably.
Yo!
For quite a few years people has wanted debug packages, but there has never really been any progress towards them. Pacman got support almost 10 years ago, and Allan wrote a POC for dbscripts in 2015[1].
In recent years this has largely been a discussion how large such a repository would become, and how to distribute it. But since we don't have any numbers things have more or less stalled with things being discussed, but never worked on. Dropping an imaginary 100 GB on some poor mirror hosts doesn't seem quite right.
However, things has been been progressing for the better when it comes to the distribution of debug symbols that can hopefully make this entire process easier for us. debuginfod has been introduced into elfutils which allows a standardized API for fetching remote debug symbols [2]. Eli and I have have been chatting a little with upstream since February to ensure we could get our package format supported. This was fairly straight forward with an example package from us and things have been working nicely.
Patches in elfutils: https://sourceware.org/git/?p=elfutils.git;a=commitdiff;h=499129e77aa88b9475... https://sourceware.org/git/?p=elfutils.git;a=commitdiff;h=0245c6ed65a80bceb1... https://sourceware.org/git/?p=elfutils.git;a=commitdiff;h=ad09e791320d131498...
We have been distributing debuginfod since 0.178-1 and debuginfod support came to gdb with the 10.1 release, which is why I'm picking up on this now :)
We did some testing with the debuginfod support with Stapelberg during the summer, and it works fairly well [3]. Eli has also started writing up the support for splitting out the debug packages into a separate pool [4]. I have since merged some of Elis dbscripts patches to the POC git migration server to test things.
The idea is to allow uploads of debug packages to repos.a.o into a separate package pool, have a public reachable debuginfod and then consider if we want/need debug package distribution. We can then check the new mirror requirements, and give a clear heads up to any potential mirror admin while providing debug packages. I think this is a reasonable compromise for everyone. OpenSUSE already has one such server as an example [5].
There isn't any one way we have to progress on this, but my proposed timeline is as follows:
- Add a debuginfod role to the infrastructure repository. - Test and deploy the dbscripts support for debug packages. - Add support to devtools for uploading debug packages. - Announce debuginfod support. - Discuss how to distribute the packages.
I anticipate we can start a new discussion for the last point as I think there is some issues we should think about in terms of archiving debug packages and so on.
Is there any questions, concerns or suggestions for this proposed implementation?
If anyone is interested trying out debug packages and debuginfod, on the POC git migration server, please do poke me!
On 23/11/2020 16:52, Morten Linderud via arch-dev-public wrote:
Give me cake, or give me debug symbols. - Some comedian, probably.Yo!
For quite a few years people has wanted debug packages, but there has never really been any progress towards them. Pacman got support almost 10 years ago, and Allan wrote a POC for dbscripts in 2015[1].
In recent years this has largely been a discussion how large such a repository would become, and how to distribute it. But since we don't have any numbers things have more or less stalled with things being discussed, but never worked on. Dropping an imaginary 100 GB on some poor mirror hosts doesn't seem quite right.
However, things has been been progressing for the better when it comes to the distribution of debug symbols that can hopefully make this entire process easier for us. debuginfod has been introduced into elfutils which allows a standardized API for fetching remote debug symbols [2]. Eli and I have have been chatting a little with upstream since February to ensure we could get our package format supported. This was fairly straight forward with an example package from us and things have been working nicely.
Patches in elfutils: https://sourceware.org/git/?p=elfutils.git;a=commitdiff;h=499129e77aa88b9475... https://sourceware.org/git/?p=elfutils.git;a=commitdiff;h=0245c6ed65a80bceb1... https://sourceware.org/git/?p=elfutils.git;a=commitdiff;h=ad09e791320d131498...
We have been distributing debuginfod since 0.178-1 and debuginfod support came to gdb with the 10.1 release, which is why I'm picking up on this now :)
We did some testing with the debuginfod support with Stapelberg during the summer, and it works fairly well [3]. Eli has also started writing up the support for splitting out the debug packages into a separate pool [4]. I have since merged some of Elis dbscripts patches to the POC git migration server to test things.
Cool!
The idea is to allow uploads of debug packages to repos.a.o into a separate package pool, have a public reachable debuginfod and then consider if we want/need debug package distribution. We can then check the new mirror requirements, and give a clear heads up to any potential mirror admin while providing debug packages. I think this is a reasonable compromise for everyone. OpenSUSE already has one such server as an example [5].
There isn't any one way we have to progress on this, but my proposed timeline is as follows:
- Add a debuginfod role to the infrastructure repository.
Please create a new ticket in the infra repo with some requirements/details! Especially with information where we host debuginfod, size requirements etc.
- Test and deploy the dbscripts support for debug packages.
Is this separate of the Git migration? Or is the git migration a requirement to get debug packages
- Add support to devtools for uploading debug packages.
- Announce debuginfod support.
- Discuss how to distribute the packages.
We can at least host them on gemini and make them staff only in the beginning.
Greetings,
Jelle
On Tue, Dec 01, 2020 at 12:58:40PM +0100, Jelle van der Waa wrote:
- Add a debuginfod role to the infrastructure repository.
Please create a new ticket in the infra repo with some requirements/details! Especially with information where we host debuginfod, size requirements etc.
I don't know about size. I think the question is if we are happy to have the debuginfod service running on gemini, or if we want another service for this purpose. This is also why I wanted some time on this on a future devops meeting :) So hopefully we can lay down some problems and solutions.
- Test and deploy the dbscripts support for debug packages.
Is this separate of the Git migration? Or is the git migration a requirement to get debug packages
This is seperate from the git migration, yes.
- Add support to devtools for uploading debug packages.
- Announce debuginfod support.
- Discuss how to distribute the packages.
We can at least host them on gemini and make them staff only in the beginning.
I agree :)
Greetings,
Jelle
Thanks for the followup!
arch-dev-public@lists.archlinux.org