[aur-general] GHC 7.8.1 packaging decisions for Arch Linux
Hello all, With the arrival of ghc 7.8.1 [0], I would like to address the following problems with a restructuring of how we treat haskell packages in archlinux: Problem 1: Updating any haskell package has been delayed until we bump ghc. Explanation: ghc is unable to produce a library that has a stable abi. In other words, if a library gets rebuilt (even if it's the same exact source), we will need to rebuild all package that depend on it, and this would in turn a messy rebuild for any kind of rebuild. Problem 2: Users are confused whether they should install packages from the repos or using cabal-install. This in turn sometimes causes them to install some packages from official repos, some from the aur, and some using cabal-install Explanation: Packages should ideally be installed from one place, either using repos/aur or cabal-install. Said that, it is entirely possible to use both repos/aur and cabal-install but for simplicity, one source is preferred. Problem 3: Our repos are unable to package every haskell package Explanation: hackage has over 5000 haskell packages in it currently. There is no way we will be able to support that many packages in our official repos without some kind of automation. And even if it would be possible, it would not necessarily be desirable. In addition, it is entirely possible to have multiple versions of the same package installed at the same time. Cabal has a very elaborate dependency solver in order to find versions that will work for all package versions installed. In addition, cabal-install 1.18 (the latest released) has added support for sandboxes, meaning that when developing haskell programs, you are now able to setup a sandbox environment that will only contain the haskell packages you want. I would like to propose the following changes to address the mentioned problems and having the ability to use sandboxes to isolate development environments. Change 1: Move every haskell related package out of [extra] into [community] except ghc and cabal-install. This includes the following 8 packages: haskell-http, haskell-mtl, haskell-network, haskell-parsec, haskell-random, haskell-text, haskell-transformers, haskell-zlib Explanation: These packages are only required to build cabal-install. Since we converted the cabal-install package to use the bootstrap script that comes with it, we no longer depend on these packages for anything in [extra]. Change 2: Make a news item stating that cabal-install is now the recommended way to install haskell packages. This wouldn't pollute the filesystem since cabal-install installs packages to the ~/.cabal directory by default. We might need to include a tip sheet about how you would handle ghc updates since it requires extra user steps. Change 3: Support users who are unable to install haskell packages that do not compile under archlinux. This would require working with the user and upstream to open up tickets and write patches for programs. At the very least we can work with the user if they do not to open up upstream bug reports and track them in our own bug tracker. There might be some packages which we would probably consider unsupported like bindings to packages that are not in the supported repos and packages that have no upstream activity and ones that are effectively unmaintained. Hopefully with the proposed changes, we can improve our user experience with using official tools to maintain haskell packages. I realize that some changes might be controversial especially change 3. This is why I posted this here in order to discuss the possibilities. I would also love to hear some opinions from vegai since he has some previous experience package haskell and has had similar thoughts in the past. Also if you're not a developer or trusted user, I would also like to hear your opinion since it would affect you also. Thanks for your time, - Tom [0] - https://github.com/ghc/ghc/releases/tag/ghc-7.8.1-release
Tom, I might come across as very critical below, but I'm really not. As you probably realise I've also thought a bit about related questions and I'm just really interested in your thoughts and answers. On Wed, Apr 9, 2014 at 7:27 AM, Thomas Dziedzic <gostrc@gmail.com> wrote:
Hello all,
With the arrival of ghc 7.8.1 [0], I would like to address the following problems with a restructuring of how we treat haskell packages in archlinux: [...] Change 1: Move every haskell related package out of [extra] into [community] except ghc and cabal-install. This includes the following 8 packages: haskell-http, haskell-mtl, haskell-network, haskell-parsec, haskell-random, haskell-text, haskell-transformers, haskell-zlib Explanation: These packages are only required to build cabal-install. Since we converted the cabal-install package to use the bootstrap script that comes with it, we no longer depend on these packages for anything in [extra].
I'm guessing this means cabal-install now is the only package outside of [community] that uses ghc to build. Is that right? Is the plan then that any future tools (i.e. non-libraries) implemented in Haskell would go into [community]? <devil advocate> There is nothing that say one HAS to wait for a ghc upgrade in order to provide newer versions of Haskell packages. As you point all that's needed is a rebuild of all the packages that depend on the upgraded one. If that's messy it sounds like you are using bad tools to handle upgrading. Are you really suggesting ArchLinux abandon packaging a whole class of software just because the tools are inadequate? </devils advocate>
Change 2: Make a news item stating that cabal-install is now the recommended way to install haskell packages. This wouldn't pollute the filesystem since cabal-install installs packages to the ~/.cabal directory by default. We might need to include a tip sheet about how you would handle ghc updates since it requires extra user steps.
It should be noted that cabal-install isn't a package manager in the true sense[1]. I'm not sure this is an argument against making the change you propose, but it's worth noting. There are quite a few other language/frameworks that have language-specific build/package systems, Python, Ruby, Perl, node.js... Are Python developers on Arch pointed towards using pip to install Python libs? I think sometimes the right thing is to point users to another package manager, e.g. packaging vim scripts for system wide installation is a bit silly, since installing a vim script affects ALL users on the system. So doing that would require providing some sort of vim-script manager to users. Then there's very little difference compared to just telling users to use Vundle/Pathogen/whatever directly instead. However, this isn't the case for Haskell/GHC...
Change 3: Support users who are unable to install haskell packages that do not compile under archlinux. This would require working with the user and upstream to open up tickets and write patches for programs. At the very least we can work with the user if they do not to open up upstream bug reports and track them in our own bug tracker. There might be some packages which we would probably consider unsupported like bindings to packages that are not in the supported repos and packages that have no upstream activity and ones that are effectively unmaintained.
How do you envision this actually working? The set of packages in [extra]/[community] is rather small today, in the order of 3 dozen, so does this mean that users are already turning to the Arch devs when they are having problems compiling Haskell packages? /M [1]: http://is.gd/vzse5G- -- Magnus Therning OpenPGP: 0xAB4DFBA4 email: magnus@therning.org jabber: magnus@therning.org twitter: magthe http://therning.org/magnus
On Wed, Apr 9, 2014 at 9:07 AM, Magnus Therning <magnus@therning.org> wrote:
Tom,
I might come across as very critical below, but I'm really not. As you probably realise I've also thought a bit about related questions and I'm just really interested in your thoughts and answers.
On Wed, Apr 9, 2014 at 7:27 AM, Thomas Dziedzic <gostrc@gmail.com> wrote:
Hello all,
With the arrival of ghc 7.8.1 [0], I would like to address the following problems with a restructuring of how we treat haskell packages in archlinux: [...] Change 1: Move every haskell related package out of [extra] into [community] except ghc and cabal-install. This includes the following 8 packages: haskell-http, haskell-mtl, haskell-network, haskell-parsec, haskell-random, haskell-text, haskell-transformers, haskell-zlib Explanation: These packages are only required to build cabal-install. Since we converted the cabal-install package to use the bootstrap script that comes with it, we no longer depend on these packages for anything in [extra].
I'm guessing this means cabal-install now is the only package outside of [community] that uses ghc to build. Is that right?
Is the plan then that any future tools (i.e. non-libraries) implemented in Haskell would go into [community]?
I would like to keep XMonad/XMobar in [community] it does seem to take up a big chunk of the haskell-* packages we have in our repos. But I've never ran into real big issues packaging haskell libraries, one minor issue is that the developers tend to oversplit packages for example haskell-data-default-* . This really makes packaging haskell libraries annoying.
It should be noted that cabal-install isn't a package manager in the
true sense[1]. I'm not sure this is an argument against making the change you propose, but it's worth noting.
There are quite a few other language/frameworks that have language-specific build/package systems, Python, Ruby, Perl, node.js... Are Python developers on Arch pointed towards using pip to install Python libs?
I think sometimes the right thing is to point users to another package manager, e.g. packaging vim scripts for system wide installation is a bit silly, since installing a vim script affects ALL users on the system. So doing that would require providing some sort of vim-script manager to users. Then there's very little difference compared to just telling users to use Vundle/Pathogen/whatever directly instead. However, this isn't the case for Haskell/GHC...
I would prefer that we don't package vim plugins or firefox extensions. Firefox has it's own extension manager and vim has a lot of solutions which work better then pacman.
Change 3: Support users who are unable to install haskell packages that do
not compile under archlinux. This would require working with the user and upstream to open up tickets and write patches for programs. At the very least we can work with the user if they do not to open up upstream bug reports and track them in our own bug tracker. There might be some packages which we would probably consider unsupported like bindings to packages that are not in the supported repos and packages that have no upstream activity and ones that are effectively unmaintained.
How do you envision this actually working? The set of packages in [extra]/[community] is rather small today, in the order of 3 dozen, so does this mean that users are already turning to the Arch devs when they are having problems compiling Haskell packages?
How many haskell developers actually use our packages in the repos/aur rather then using caba-install?
-- Jelle van der Waa
On Wed, Apr 9, 2014 at 12:35 AM, Jelle van der Waa <jelle@vdwaa.nl> wrote:
I would like to keep XMonad/XMobar in [community] it does seem to take up a big chunk of the haskell-* packages we have in our repos. But I've never ran into real big issues packaging haskell libraries, one minor issue is that the developers tend to oversplit packages for example haskell-data-default-* . This really makes packaging haskell libraries annoying.
I don't want to prevent people from maintaining haskell packages in supported repos. I just want to make cabal-install a path with which people don't have to think twice about.
I would prefer that we don't package vim plugins or firefox extensions. Firefox has it's own extension manager and vim has a lot of solutions which work better then pacman
I also share this belief about haskell. Both pacman and cabal-install have their own pros and cons, but for my personally, I find that cabal-install has more benefit to me personally for haskell packages.
How many haskell developers actually use our packages in the repos/aur rather then using caba-install?
I can only speak from personal experience, but I have used cabal-install exclusively for a couple of years now not just for development but for also installing tools like xmonad. My guess is that given the limited supply of packages in our repos, I can guestimate 0 use it for development but there might be a lot of users that actually use the haskell tools like xmonad.
On Wed, Apr 9, 2014 at 12:07 AM, Magnus Therning <magnus@therning.org>wrote:
I'm guessing this means cabal-install now is the only package outside of [community] that uses ghc to build. Is that right?
That would be correct.
Is the plan then that any future tools (i.e. non-libraries) implemented in Haskell would go into [community]?
This would also be correct. I believe that most people who use packages in our supported repos don't actually use the haskell libraries themselves, but rather the tools that depend on them. (e.g. xmonad) I am not against keeping these tools around and their dependencies if someone wants to maintain them, but I personally have no interest in maintaining them myself.
<devil advocate> There is nothing that say one HAS to wait for a ghc upgrade in order to provide newer versions of Haskell packages. As you point all that's needed is a rebuild of all the packages that depend on the upgraded one. If that's messy it sounds like you are using bad tools to handle upgrading. Are you really suggesting ArchLinux abandon packaging a whole class of software just because the tools are inadequate? </devils advocate>
I am aware that the way we are packaging haskell packages is non optimal and could be automated away. I already have a tool that automates the rebuild of [extra] packages, but since I don't/want to maintain any packages that aren't dependencies of cabal-install, I haven't looked into automating those builds. I don't want to focus solely on this point though because I just felt like I should make people aware of this problem. This isn't the only reason why I want to move away from haskell packages in pacman. It should be noted that cabal-install isn't a package manager in the
true sense[1]. I'm not sure this is an argument against making the change you propose, but it's worth noting.
I agree. Cabal does lack even basic features that other people would expect from a package manager. For example, upgrades, removal. Although there are ways to have the same effect as upgrading since you can have multiple versions of the same package installed, you could just install the same package after a cabal update. That's why I suggested writing up a tip sheet on basic cabal usage for people who aren't as used to cabal as others.
There are quite a few other language/frameworks that have language-specific build/package systems, Python, Ruby, Perl, node.js... Are Python developers on Arch pointed towards using pip to install Python libs?
I think that the languages you mentioned could be split up into 2 categories. Python and Perl have a large number of interested developers and trusted users maintaining those packages. Python has over 700 packages, and Perl has over 500 packages and so, there is a lot of interest in those languages. On the other hand, you have Haskell, Ruby, and Node.Js where they have a smaller following. Haskell has about 90 packages (including i686/x86_64), Ruby has about 60, and Node.Js has approximately none. The popular languages have a critical mass which I believe lets people get away with using the pacman package manager. So for languages like python and perl, it might make sense to use pacman. But for languages like haskell, ruby, and node.js I believe using the system package manager leaves a lot to be desired.
How do you envision this actually working?
I would like to officially announce that there are 2 paths of maintaining haskell packages on your system. One would be using cabal-install and I would encourage users to try using it. The other would be to use whatever is in the supported repositories. I would also like to officially announce that one or the other is supported, but they should be mutually exclusive.
The set of packages in [extra]/[community] is rather small today, in the order of 3 dozen, so does this mean that users are already turning to the Arch devs when they are having problems compiling Haskell packages?
No. I believe that most users actually know what the correct path is. I just want to make it officially announced rather than having any potential confusion on which path to take and which paths are supported.
On 04/09/2014 01:27 AM, Thomas Dziedzic wrote:
Problem 2: Users are confused whether they should install packages from the repos or using cabal-install. This in turn sometimes causes them to install some packages from official repos, some from the aur, and some using cabal-install Explanation: Packages should ideally be installed from one place, either using repos/aur or cabal-install. Said that, it is entirely possible to use both repos/aur and cabal-install but for simplicity, one source is preferred.
Doesn't every language with its own package manager have this problem? For example, Python. Is there a good solution? Users knowing about this issue and making their own decisions is the current solution on every distro I'm familiar with.
Change 2: Make a news item stating that cabal-install is now the recommended way to install haskell packages. This wouldn't pollute the filesystem since cabal-install installs packages to the ~/.cabal directory by default. We might need to include a tip sheet about how you would handle ghc updates since it requires extra user steps.
This is the way I do it on my personal computer, FWIW. If there were any packages in the repo written in Haskell that are used by non-Haskellers, then all those packages' deps would have to be in the repos too. (e.g. darcs and pandoc, which are in AUR currently). Cheers, -Isaac
Doesn't every language with its own package manager have this problem? For example, Python. Is there a good solution? Users knowing about this issue and making their own decisions is the current solution on every distro I'm familiar with.
Aye. There are some upshots to allowing users to install software via either the system package manager or a language-specific one, instead of forcing them down a single path. For example, with ruby: tools such as bundler, rbenv and rvm provide users with some awesome capabilities, such as being able to install multiple versions of Ruby and multiple versions of gems, either system-wide, in a user's home directory or in a custom directory. That's very useful, and the user should have that option - but if the user wants a simpler, more straightforward option, they can use `gem` or `pacman`. (And the choice between gem and pacman has its own tradeoffs.) Couldn't the same be said of Haskell packages? Using pacman to manage haskell packages provides some niceties, such as automatic updates (whenever -Syu is performed), whereas cabal-install provides its own niceties, such as access to numerous packages. So... is there anything wrong with letting the user choose the tool that suits their needs?
On Thu, Apr 10, 2014 at 7:20 PM, Jeremy Audet <ichimonji10@gmail.com> wrote:
Doesn't every language with its own package manager have this problem? For example, Python. Is there a good solution? Users knowing about this issue and making their own decisions is the current solution on every distro I'm familiar with.
Aye. There are some upshots to allowing users to install software via either the system package manager or a language-specific one, instead of forcing them down a single path. For example, with ruby: tools such as bundler, rbenv and rvm provide users with some awesome capabilities, such as being able to install multiple versions of Ruby and multiple versions of gems, either system-wide, in a user's home directory or in a custom directory. That's very useful, and the user should have that option - but if the user wants a simpler, more straightforward option, they can use `gem` or `pacman`. (And the choice between gem and pacman has its own tradeoffs.)
Couldn't the same be said of Haskell packages? Using pacman to manage haskell packages provides some niceties, such as automatic updates (whenever -Syu is performed), whereas cabal-install provides its own niceties, such as access to numerous packages. So... is there anything wrong with letting the user choose the tool that suits their needs?
I guess I should modify my wording for Change 2 to be: "Change 2: Make a news item stating that either cabal-install or pacman packages are now the recommended way to install haskell packages. Not both in one system."
On 2014-04-08 22:27, Thomas Dziedzic wrote:
Hello all,
With the arrival of ghc 7.8.1 [0], I would like to address the following problems with a restructuring of how we treat haskell packages in archlinux:
Problem 1: Updating any haskell package has been delayed until we bump ghc. Explanation: ghc is unable to produce a library that has a stable abi. In other words, if a library gets rebuilt (even if it's the same exact source), we will need to rebuild all package that depend on it, and this would in turn a messy rebuild for any kind of rebuild.
Changing depends on packages to exact versions will make these incompatibilities rise quickly. Eg: package A should depend on packageB=x.y.z-n That would avoid mixing up different versions. This would avoid a library being updated in a system until all packages that depend on it have been rebuilt. -- Hugo Osvaldo Barrera
participants (6)
-
Hugo Osvaldo Barrera
-
Isaac Dupree
-
Jelle van der Waa
-
Jeremy Audet
-
Magnus Therning
-
Thomas Dziedzic