[arch-dev-public] Tracking different CPU architectures with pkgstats

older
[arch-dev-public] Killing python2...

Pierre Schmitz

7 Mar 2021 7 Mar '21

1:40 p.m.

Hi all, while reading Allan's RFC about increasing Arch's CPU requirements(*) I had the idea to start tracking the different x86_64 architecture level using pkgstats. I am sure whether to drop support for old CPUs is not a matter of "if" but "when"; similar as it was with i686. Therefore it should help to have a rough estimate about which CPUs our users have and how the numbers develop over time. While I am at it, I'd like to also support different ARM architectures provided by archlinuxarm.org (or maybe even i486/i686 by archlinux32.org). There is a small catch though and the reason I am asking for your opinion: While we will know how many users use which architecture, all package usage data will be aggregated regardless of architecture. This means for a given package we cannot tell how often it is used on a specific architecture compared to others. I do not think that this would matter for most packages though. Greetings, Pierre *) https://gitlab.archlinux.org/archlinux/rfcs/-/blob/c5ee0eb715a01c6cd10aa5a9d...

Show replies by date

Eli Schwartz

7 Mar 7 Mar

3:25 p.m.

On 3/7/21 8:40 AM, Pierre Schmitz via arch-dev-public wrote:

...

While I am at it, I'd like to also support different ARM architectures provided by archlinuxarm.org (or maybe even i486/i686 by archlinux32.org). There is a small catch though and the reason I am asking for your opinion: While we will know how many users use which architecture, all package usage data will be aggregated regardless of architecture. This means for a given package we cannot tell how often it is used on a specific architecture compared to others. I do not think that this would matter for most packages though.

Given we should anyways be building the entire distro for any officially supported architecture, I'd say it won't matter at all. I'd definitely be interested in pkgstats for architectures. -- Eli Schwartz Bug Wrangler and Trusted User

Levente Polyak

9 Mar 9 Mar

6:35 p.m.

Hi Pierre, I think at this point we should actually convert the "pet project" into an real official service hosted under the archlinux.org hood :) cheers, Levente

Pierre Schmitz

14 Mar 14 Mar

2:07 p.m.

I just updated the server to also accept x86_64 feature levels, ARM and even i686. There is a new version of pkgstats (>= 3.1.0; currently in [testing]) which is able to detect feature levels. ARM support is pretty early, but x86_64 should be fine (using Intel's cpuid library). If you like to check what gets detected on your system run: $ pkgstats submit --dump-json | head system.architecture is your CPU and os.architecture should be the same as "uname -m" Let me know if this does work for you and especially if it does not. Using Qemu for testing is quite limited and I lack old, new and AMD CPUs. An API and UI to analyze these data will follow in the future. (I guess we need to wait a few weeks to see some valid results) Greetings, Pierre On Sun, Mar 7, 2021 at 2:40 PM Pierre Schmitz wrote:

...

Hi all,

while reading Allan's RFC about increasing Arch's CPU requirements(*) I had the idea to start tracking the different x86_64 architecture level using pkgstats. I am sure whether to drop support for old CPUs is not a matter of "if" but "when"; similar as it was with i686. Therefore it should help to have a rough estimate about which CPUs our users have and how the numbers develop over time.

While I am at it, I'd like to also support different ARM architectures provided by archlinuxarm.org (or maybe even i486/i686 by archlinux32.org). There is a small catch though and the reason I am asking for your opinion: While we will know how many users use which architecture, all package usage data will be aggregated regardless of architecture. This means for a given package we cannot tell how often it is used on a specific architecture compared to others. I do not think that this would matter for most packages though.

Greetings,

Pierre

*) https://gitlab.archlinux.org/archlinux/rfcs/-/blob/c5ee0eb715a01c6cd10aa5a9d...

Levente Polyak

2:16 p.m.

On 3/14/21 3:07 PM, Pierre Schmitz via arch-dev-public wrote:

...

I just updated the server to also accept x86_64 feature levels, ARM and even i686. There is a new version of pkgstats (>= 3.1.0; currently in [testing]) which is able to detect feature levels. ARM support is pretty early, but x86_64 should be fine (using Intel's cpuid library).

If you like to check what gets detected on your system run: $ pkgstats submit --dump-json | head system.architecture is your CPU and os.architecture should be the same as "uname -m"

Let me know if this does work for you and especially if it does not. Using Qemu for testing is quite limited and I lack old, new and AMD CPUs.

An API and UI to analyze these data will follow in the future. (I guess we need to wait a few weeks to see some valid results)

Hi Pierre, that sounds wonderful, thanks for the work, this will be nice data points :) Did you see my previous mail? It would be amazing if you can consider growing this side-project into something official in terms of being available on http://pkgstats.archlinux.org/ I think this is really a great idea and project that we should advocate in the official hosting namespace :) cheers and thanks, Levente

Sven-Hendrik Haase

15 Mar 15 Mar

9:42 p.m.

On Sun, 14 Mar 2021 at 15:17, Levente Polyak via arch-dev-public wrote:

...

On 3/14/21 3:07 PM, Pierre Schmitz via arch-dev-public wrote:

...
I just updated the server to also accept x86_64 feature levels, ARM and even i686. There is a new version of pkgstats (>= 3.1.0; currently in [testing]) which is able to detect feature levels. ARM support is pretty early, but x86_64 should be fine (using Intel's cpuid library).

If you like to check what gets detected on your system run: $ pkgstats submit --dump-json | head system.architecture is your CPU and os.architecture should be the same as "uname -m"

Let me know if this does work for you and especially if it does not. Using Qemu for testing is quite limited and I lack old, new and AMD CPUs.

An API and UI to analyze these data will follow in the future. (I guess we need to wait a few weeks to see some valid results)

Hi Pierre,

that sounds wonderful, thanks for the work, this will be nice data points :)

Did you see my previous mail? It would be amazing if you can consider growing this side-project into something official in terms of being available on http://pkgstats.archlinux.org/

I think this is really a great idea and project that we should advocate in the official hosting namespace :)

cheers and thanks, Levente

I second this and we've tried to touch on this in the past. I think "officializing" pkgstats would be neat. Cheers, Sven

Pierre Schmitz

16 Mar 16 Mar

5 p.m.

Hi, I saw your post; I just did not want to go off-topic right away. :-) At least as long as I am the only contributor I hesitate to give up on how the infrastructure is set up. It seems to be much more complex to do this on "official" servers. I like being able to switch things around outside of just deploying some scripts. E.g. I do automatic deployments, updates, change Web server or database configurations etc.. Greetings, Pierre On Mon, Mar 15, 2021 at 10:42 PM Sven-Hendrik Haase via arch-dev-public wrote:

...

On Sun, 14 Mar 2021 at 15:17, Levente Polyak via arch-dev-public wrote:

...
On 3/14/21 3:07 PM, Pierre Schmitz via arch-dev-public wrote:

...
I just updated the server to also accept x86_64 feature levels, ARM and even i686. There is a new version of pkgstats (>= 3.1.0; currently in [testing]) which is able to detect feature levels. ARM support is pretty early, but x86_64 should be fine (using Intel's cpuid library).

If you like to check what gets detected on your system run: $ pkgstats submit --dump-json | head system.architecture is your CPU and os.architecture should be the same as "uname -m"

Let me know if this does work for you and especially if it does not. Using Qemu for testing is quite limited and I lack old, new and AMD CPUs.

An API and UI to analyze these data will follow in the future. (I guess we need to wait a few weeks to see some valid results)

Hi Pierre,

that sounds wonderful, thanks for the work, this will be nice data points :)

Did you see my previous mail? It would be amazing if you can consider growing this side-project into something official in terms of being available on http://pkgstats.archlinux.org/

I think this is really a great idea and project that we should advocate in the official hosting namespace :)

cheers and thanks, Levente

I second this and we've tried to touch on this in the past. I think "officializing" pkgstats would be neat.

Cheers, Sven

Levente Polyak

7:27 p.m.

On 3/16/21 6:00 PM, Pierre Schmitz via arch-dev-public wrote:

...

Hi, I saw your post; I just did not want to go off-topic right away. :-) At least as long as I am the only contributor I hesitate to give up on how the infrastructure is set up. It seems to be much more complex to do this on "official" servers. I like being able to switch things around outside of just deploying some scripts. E.g. I do automatic deployments, updates, change Web server or database configurations etc..

Hey Pierre :) Hm I kind of get the way you feel... but i really really believe it would be very beneficial if we make this an official project. I don't think there needs to be a strong relation between contributor and hosting, we also host foreign software we don't even write ourselves. Otherwise every main dev of one of our projects would just host their stuff on their private servers as well :D I don't think you need to have any fear about the complexity or losing control, it is actually pretty straight forward and the devops team is more then open to implement every single bit of the devops ansible infra setup. So there won't be any workload on you :) There are multiple ways, for the security tracker (which I'm also more or less basically the only main dev :P) it is as easy as just bumping the commit hash [0]. I kind of understand your "concern", if you aim more for a fluent deployment, we have also prototyped GitLab CI pipeline deployment for the AUR in a convenient and secure way. This can give more deployment control without the need to always touch ansible. I'm a hundred percent convinced there is a good way to make all parties happy and still be able to promote your side project into an awesome official bit of Arch Linux as it truly deserves it. sincerely, Levente PS: we have a bottom reply post policy on our mailinglist :D :D :D [0] https://gitlab.archlinux.org/archlinux/infrastructure/-/commit/61869fdd16fe5...

1137

Age (days ago)

1146

Last active (days ago)

List overview

Download

7 comments

4 participants

participants (4)

Eli Schwartz
Levente Polyak
Pierre Schmitz
Sven-Hendrik Haase

[arch-dev-public] Tracking different CPU architectures with pkgstats

tags

participants (4)