On Sat, Dec 18, 2010 at 11:26 PM, Dan McGee wrote:

On Sat, Dec 18, 2010 at 11:20 PM, Allan McRae wrote:

...

On 19/12/10 04:59, Dieter Plaetinck wrote:

...

Last time the topic "moving nilfs/btrfs utilities in core" generated some discussion, but no outcome. (and not even all my questions were answered)

(http://mailman.archlinux.org/pipermail/arch-dev-public/2010-December/018656....)

So I'll try again, and I'll try to be more clear.

question 1: If you install arch linux on a foo-filesystem, should you install foo-utils as well? so that at least your system doesn't break when it needs to run fsck.foo during boot.  IMHO the answer is yes.

Yes.

...

question 2: what exacly is the definition of the core repo? Pierre tells me "The idea of core was to provide a minimal set of packages that are needed by nearly all users to set up a base system."  If is true, what separates base from core? Because this definition looks an awful lot like the definition of base to me.

AFAIK core is about bundling all packages which are critical (for the purpose of getting the system to run) for some users, depending on their particular setup.  This is the reasoning why we include the core repo on our core install images, because we say "with this repo, users can install what they need in order to make their system boot and run fine". This is also in line with what I see on https://wiki.archlinux.org/index.php/Official_Repositories ("will provide you with a fully functional base system")

This is the problem: - we want to allow users to use new filesystems during installation.  Currently this includes nilfs2 and btrfs.   (marked as experimental in aif). - On the other hand we apparently don't allow infrequently used packages in core. Even if I would wait and only support   these filesystems when they are declared stable, their package usage would still be low.  It would be pretty backwards if I would need   to wait for users to configure their systems using the filesystems manually, so the package usage goes up, before I can include them   on the installation media.   Currently, dosfstools, nilfs-utils and btrfs-progs-unstable are not in core and can't be installed by using a networkless install with the core-images.

So, How do we solve this?  I see two outcomes: - we add the packages mentioned above to core, and loosen up the signoff requirements for packages with low usage.   (because "commonly used" and "needed for a base system" are very different things) - we do not add these packages to core, and add something about "must have high usage" on the wikipage i mentioned above.   this also means I will need to include a small repo with "needed for base systems, but with low usage" on the core install images, to   allow users to install these packages.  And I'll need to adapt aif because it now needs to use additional repositories to install   required-to-boot packages.

I vote for outcome 1.

Basically I don't care what the outcome is here.   But note that we can probably safely assume that reiserfs is more commonly used than both the two filesystems that you mention and I have yet to receive a signoff for reiserfsprogs (posted 2010-12-04...).   So moving the packages for nilfs/btrfs to [core] will only serve to delay updates...  Then again, I see no reason why these packages should not be in [core].

My solution would be more like your #2 suggestion.  Instead of a "core" CD, make an "install" CD which has a set of packages you decide on it. This would be largely based on [core], but you can include packages from [extra] then too.  You can also drop some [core] packages from the CD. For example, there is no point putting gcc-{fortran,objc,ada} on the install CD at all.  They are only in [core] because split packages can not go across repos.  So get rid of them and gain 40MB more space for your dual arch iso.

I'm of the opposite opinion on this one; I think #1 makes more sense. We're looking at it backwards- "this package can't go in core because it is hard to get signoffs" vs. "perhaps we can adjust our signoff policy for lower-usage core packages". Filesystem utilities, at least to me, seem like the definition of what core should contain. Reiser/JFS/XFS/BTRFS/NILFS, while not the top filesystems in use in Linux, deserve to be supported "out of the box" by us.

For a minor bit of history, "core" is only named the way it is because it linguistically makes sense with a repository named "extra". core was never meant to be anything more or less than "current" used to be - which was a repo full of software that the developers hand-picked to be important. That doesn't answer the question, though. Now-a-days, core is treated as if it were the mission-critical parts of a running system. That's a good definition, but it also doesn't help answer the filesystem util question. Are they critical to a running system? Not exactly... once I have an installed filesystem, I don't need them anymore (it's a good *suggestion*, but it's not *critical*). So what do we gain with having the filesystem utils in core? Signoffs, and in theory a more stable installer. But it already appears that signoffs for the more esoteric filesystem utils are hard to get. In my eyes, this seems to make the benefit of gaining signoffs moot. It seems this question is really about the following: should things needed during install time be part of core? Or rather, is installation an important enough part of Arch Linux to require the (theoretical) added stability of the signoff process? I don't have a real concrete answer. I wish I could give one. It boils down to this: do install-time necessities need to be in core?

On Wed, 22 Dec 2010 19:14:19 +0100, Dieter Plaetinck wrote:

I realise I just suggested some pretty radical changes, but in my mind they make a lot of sense. I hope I didn't miss some obvious downsides (please point them out), I will (and I hope you do too) spend some nights pondering about this.

I think the core repo and its sign-off policy has proven itself to be a good idea. But we could make a little adjustment to our signoff policy to solve your filesystem package problem. [core]: This contains everything you need to boot up, connect to the internet and install additional packages from e.g. [extra] base group: A smaller subset of [core] that include packages that should be installed on every Arch system. base-devel group: Additional packages needed to build our base packages. (This group is indeed questionable and one might consider moving them to [extra] Everything else in core are packages that are not needed by everybody but required by some to "boot up, connect to the internet and install additional packages"; e.g. file system packages, firmware for your wireless card, wireless_tools etc.. I would suggest to change our policy to this: * packages in the base group and its dependencies still need the usual two sign-off per architecture * sign-offs for all other packages in core are optional; they still need to enter testing first, but can be moved to core without any sign-off after 3 days (or one week or whatever) The install CD would than contain the full core repo. What do you think of this proposal? Greetings, Pierre -- Pierre Schmitz, https://users.archlinux.de/~pierre

Am 22.12.2010 20:26, schrieb Pierre Schmitz:

I think the core repo and its sign-off policy has proven itself to be a good idea. But we could make a little adjustment to our signoff policy to solve your filesystem package problem.

Agreed.

[core]: This contains everything you need to boot up, connect to the internet and install additional packages from e.g. [extra]

Sorry for being late on this, but this is what I remember from when we redefined the repos - [current] and [extra] was not completely well-defined, so we came up with this (which is mostly what Pierre said): [core] was meant to be the core of Arch, which was supposed to be on the installer. This means: * Packages that are needed to boot (C library, kernel, init scripts, filesystem, system logger, at least one bootloader). * Packages that may be needed to connect to the internet (dhcp client, wireless management tools, pp(t)p, various common VPN clients) - for installing more packages. * Essential package building: default compiler, fakeroot, other support tools for makepkg * Packages for file system management: mkfs and fsck tools for common file systems (this would include btrfs tools, if btrfs is popular enough). * Packages that do not match any of the above criteria, but that virtually anyone will want or need early in the system setup process. The only one I can think of is openssh. * Dependencies (but not necessarily makedepends) of the above. This was never written down this explicitly, but this is how I remember the consensus we made back then. tpowa and andy were iirc two of the driving forces behind the repository transition, I compiled the initial list of packages.

base group: A smaller subset of [core] that include packages that should be installed on every Arch system.

That was the idea. Apart from wpa_supplicant, this is okay the way it is (wpa_supplicant is afaik still in base, but shouldn't be).

base-devel group: Additional packages needed to build our base packages. (This group is indeed questionable and one might consider moving them to [extra]

The idea was to run 'pacman -S base-devel' and have all essential support tools for makepkg. I'd keep them in core - at least gcc will stay in core, unless we want to make two separate PKGBUILDs for gcc and gcc-libs. Having them in core is a good idea, as someone might want to do a core install and makepkg something before being able to continue (to compile an important network access tool).

Everything else in core are packages that are not needed by everybody but required by some to "boot up, connect to the internet and install additional packages"; e.g. file system packages, firmware for your wireless card, wireless_tools etc..

Agreed in principle, but I went into the details more above.

I would suggest to change our policy to this: * packages in the base group and its dependencies still need the usual two sign-off per architecture

One sign-off is implicit by the package builder, so we always had one extra signoff from someone else. I would also only require one sign-off for -any packages, instead of one per architecture.

* sign-offs for all other packages in core are optional; they still need to enter testing first, but can be moved to core without any sign-off after 3 days (or one week or whatever)

Nice idea.

The install CD would than contain the full core repo.

As was originally intended.

What do you think of this proposal?

+1

On Sun, Dec 19, 2010 at 12:26 AM, Dan McGee wrote:

On Sat, Dec 18, 2010 at 11:20 PM, Allan McRae wrote:

...

On 19/12/10 04:59, Dieter Plaetinck wrote:

...

Last time the topic "moving nilfs/btrfs utilities in core" generated some discussion, but no outcome. (and not even all my questions were answered)

(http://mailman.archlinux.org/pipermail/arch-dev-public/2010-December/018656....)

So I'll try again, and I'll try to be more clear.

question 1: If you install arch linux on a foo-filesystem, should you install foo-utils as well? so that at least your system doesn't break when it needs to run fsck.foo during boot.  IMHO the answer is yes.

Yes.

...

question 2: what exacly is the definition of the core repo? Pierre tells me "The idea of core was to provide a minimal set of packages that are needed by nearly all users to set up a base system."  If is true, what separates base from core? Because this definition looks an awful lot like the definition of base to me.

AFAIK core is about bundling all packages which are critical (for the purpose of getting the system to run) for some users, depending on their particular setup.  This is the reasoning why we include the core repo on our core install images, because we say "with this repo, users can install what they need in order to make their system boot and run fine". This is also in line with what I see on https://wiki.archlinux.org/index.php/Official_Repositories ("will provide you with a fully functional base system")

This is the problem: - we want to allow users to use new filesystems during installation.  Currently this includes nilfs2 and btrfs.   (marked as experimental in aif). - On the other hand we apparently don't allow infrequently used packages in core. Even if I would wait and only support   these filesystems when they are declared stable, their package usage would still be low.  It would be pretty backwards if I would need   to wait for users to configure their systems using the filesystems manually, so the package usage goes up, before I can include them   on the installation media.   Currently, dosfstools, nilfs-utils and btrfs-progs-unstable are not in core and can't be installed by using a networkless install with the core-images.

So, How do we solve this?  I see two outcomes: - we add the packages mentioned above to core, and loosen up the signoff requirements for packages with low usage.   (because "commonly used" and "needed for a base system" are very different things) - we do not add these packages to core, and add something about "must have high usage" on the wikipage i mentioned above.   this also means I will need to include a small repo with "needed for base systems, but with low usage" on the core install images, to   allow users to install these packages.  And I'll need to adapt aif because it now needs to use additional repositories to install   required-to-boot packages.

I vote for outcome 1.

Basically I don't care what the outcome is here.   But note that we can probably safely assume that reiserfs is more commonly used than both the two filesystems that you mention and I have yet to receive a signoff for reiserfsprogs (posted 2010-12-04...).   So moving the packages for nilfs/btrfs to [core] will only serve to delay updates...  Then again, I see no reason why these packages should not be in [core].

My solution would be more like your #2 suggestion.  Instead of a "core" CD, make an "install" CD which has a set of packages you decide on it. This would be largely based on [core], but you can include packages from [extra] then too.  You can also drop some [core] packages from the CD. For example, there is no point putting gcc-{fortran,objc,ada} on the install CD at all.  They are only in [core] because split packages can not go across repos.  So get rid of them and gain 40MB more space for your dual arch iso.

I'm of the opposite opinion on this one; I think #1 makes more sense. We're looking at it backwards- "this package can't go in core because it is hard to get signoffs" vs. "perhaps we can adjust our signoff policy for lower-usage core packages". Filesystem utilities, at least to me, seem like the definition of what core should contain. Reiser/JFS/XFS/BTRFS/NILFS, while not the top filesystems in use in Linux, deserve to be supported "out of the box" by us.

-Dan

I agree. A while ago, I suggested a way to improve signoff for low usage packages. But it died because of lack of interest. I'll repeat it here. Basically, the idea was be to have the devs fill out a list (this hasn't been updated in a while): https://wiki.archlinux.org/index.php/DeveloperWiki:CoreSignoffs#List_of_pote... Unfortunately, very few devs bothered filling it out when I created it, so the process died. This will tell us what are the low usage packages and how many (and which) devs can give signoff. This will prevent us having package in signoff limbo because not enough devs use them. This might encourage signoffs by getting rid of the "Someone else will signoff" syndrom. For these low usage packages: 1 - the maintainer could also sent the signoff thread to the arch-general ML and invite users for signoff. I've already been doing this for lvm2. 2- we could have a policy so that after a certain amount of days (7 days???) in testing and no bug/problem reported, the package can be moved to core. The maintainer could bump the thread about halfway through. This will eliminate having packages in testing in weeks waiting for signoffs and will give a clear guideline for maintainers of these packages. This could be easily done if there's interest. It's just a question of updating the list, have the devs fill it out and decide the minimal time these packages should remain in testing. Eric

On Fri, Dec 24, 2010 at 8:02 AM, Allan McRae wrote:

On 23/12/10 05:27, Eric Bélanger wrote:

...

2- we could have a policy so that after a certain amount of days (7 days???) in testing and no bug/problem reported, the package can be moved to core. The maintainer could bump the thread about halfway through.  This will eliminate having packages in testing in weeks waiting for signoffs and will give a clear guideline for maintainers of these packages.

Don't we sort of have that policy anyway, although maybe not officially... Aaron used to give a "signoff" on any package that had been in [testing] for a week or two without signoffs and bug reports.

Yes, but not officially AFAIK. It's currently at the maintainer's discretion.

But an official policy saying that after 7 days a "low usage" package can be moved from [testing] provided they do not have a new bug report would be useful.  I'd define "low usage" as anything not expected to be on most devs installs.

Allan

If it would be too much trouble to have everyone put their name beside the packages they can signoff on my original list, someone (I could do it but not after the New Year) could just make a list of what they believe is low usage packages and submit it to this list for discussion and approval.