Re: [arch-general] [arch-dev-public] Ruby directory clean up proposal.
Hi, I'm not a dev, so am replying on arch-general, but a TU who uses a lot of ruby on Arch. Thanks for thinking about this. I too have been trying to come up with my own sane way of using ruby - gems particularly - with Arch. On Sunday 12 Feb 2012 17:37:17 Thomas Dziedzic wrote:
Current layout: /usr/lib/ruby/site_ruby - packages either installed here /usr/lib/ruby/gems/[ruby_base_version] - or here, this directory contains both pacman installed packages and packages installed using the gem tool.
Planned layout: /usr/lib/ruby/site_ruby - manually installed files by user go here, packages shouldn't touch this /usr/lib/ruby/gems/[ruby_base_version] - pacman packages built from gems go here /usr/lib/ruby/vendor_ruby - ruby packages installed with pacman which aren't gems go here $HOME/.gem/ruby/[ruby_base_version] - used by the gem command to install packages unmanaged by pacman
+1 from me, except that we should probably emphasise that /usr/lib/ruby/site_ruby really is for site-specific stuff. Anything that is generic enough to be able to go in a pacman package (and/or a gem) should do.
/etc/gemrc - contains "gem: --user-install" to install user installed gems with gem to $HOME/.gem/gems
I didn't know about --user-install, but I just set GEM_HOME (actually, I use RVM). Can what you want be done by globally setting GEM_HOME to be based off of $HOME? Or is --user-install the preferred way?
If the user chooses to install gems using gem, they will have to add the bin directory to the $PATH: export PATH="$PATH:$(ruby -rubygems -e 'puts Gem.user_dir')/bin".
Yep, or just something derived from $GEM_HOME if we were to use that approach instead.
Benefits of this change include: Using the proper directory layout introduced in ruby 1.9 (using vendor_ruby). Clean separation of pacman installed packages (/usr/lib/ruby/gems/[ruby_base_version] & /usr/lib/ruby/vendor_ruby) from gem installed packages ($HOME/.gem/ruby/[ruby_base_version]).
Problems left: The user will still be able to install to the system wide gem directory /usr/lib/ruby/gems/[ruby_base_version] if they remove --user-install from /etc/gemrc
Possible remedies: 1) Leave it as is. If the user runs gem with --no-user-install then gem will still install the files to the system wide directory at /usr/lib/ruby/gems/[ruby_base_version] and will install binaries to /usr/bin This should provide a clean default and is the easiest path. This option is still better than the one we currently use where we try to install to the system wide location by default.
I think it's worth separating out the "user" and the "admin" in this argument. To install a gem system-wide, you have to do something like "sudo gem install XXX", right? This is almost always a bad idea, IMO, and people hopefully won't do it at least during the normal run of working with ruby. Personally, I don't let gem mess with anything outside my home directory. And plain "gem install XXX" will not accidentally install system-wide, right? Ruby is quite good for separating between development and deployment environments - it seems to make sense for us to do a similar thing here, seeing the Arch OS layer itself as a particular kind of deployment, where things are managed by pacman.
2) Change all ruby pacman packages to use local _gemdir='/usr/lib/ruby/vendor_ruby' This would free up /usr/lib/ruby/gems/[ruby_base_version] which we could move to /var/lib/ruby/gems/[ruby_base_version] for system wide gems installed with the gem command.
And as much as I have to admit that I'd really like to be able to install gems system-wide and use their built-in dependency management, bundler etc... I think that road probably leads to madness and death, especially when many popular gems will still be managed by pacman (and rightly so). So, I would be careful about doing this, and would suggest that we go for only letting 'gem' install things under $HOME. Anything that should go system-wide should have a PKGBUILD.
I'm currently leaning towards 1 since it will be a simple solution with no changes needed to existing ruby-* packages.
Yep. Of course, suggesting RVM for user-installed gems(ets) is probably also a good idea. Pete. PS. A while ago I was getting very frustrated that the version of rubygems bundled with ruby itself was so out-of-date so quickly, but 'gem update --system' is a very bad idea in combination with pacman. So, I spent a while hacking together this package, which installs the latest version of rubygems itself alongside ruby, system-wide: https://aur.archlinux.org/packages.php?ID=50346 It's hacky, but it works :-) I wish there was a cleaner way.
On Mon, Feb 13, 2012 at 7:53 PM, Peter Lewis
Hi,
I'm not a dev, so am replying on arch-general, but a TU who uses a lot of ruby on Arch.
Thanks for thinking about this. I too have been trying to come up with my own sane way of using ruby - gems particularly - with Arch.
On Sunday 12 Feb 2012 17:37:17 Thomas Dziedzic wrote:
Current layout: /usr/lib/ruby/site_ruby - packages either installed here /usr/lib/ruby/gems/[ruby_base_version] - or here, this directory contains both pacman installed packages and packages installed using the gem tool.
Planned layout: /usr/lib/ruby/site_ruby - manually installed files by user go here, packages shouldn't touch this /usr/lib/ruby/gems/[ruby_base_version] - pacman packages built from gems go here /usr/lib/ruby/vendor_ruby - ruby packages installed with pacman which aren't gems go here $HOME/.gem/ruby/[ruby_base_version] - used by the gem command to install packages unmanaged by pacman
+1 from me, except that we should probably emphasise that /usr/lib/ruby/site_ruby really is for site-specific stuff. Anything that is generic enough to be able to go in a pacman package (and/or a gem) should do.
/etc/gemrc - contains "gem: --user-install" to install user installed gems with gem to $HOME/.gem/gems
I didn't know about --user-install, but I just set GEM_HOME (actually, I use RVM). Can what you want be done by globally setting GEM_HOME to be based off of $HOME? Or is --user-install the preferred way?
I did look into $GEM_HOME but I thought this was inadequate since running sudo gem install foo would still install foo to the system-wide gem directory. Having --user-install in /etc/gemrc forces it to install to /root/.gem/gems instead.
If the user chooses to install gems using gem, they will have to add the bin directory to the $PATH: export PATH="$PATH:$(ruby -rubygems -e 'puts Gem.user_dir')/bin".
Yep, or just something derived from $GEM_HOME if we were to use that approach instead.
Benefits of this change include: Using the proper directory layout introduced in ruby 1.9 (using vendor_ruby). Clean separation of pacman installed packages (/usr/lib/ruby/gems/[ruby_base_version] & /usr/lib/ruby/vendor_ruby) from gem installed packages ($HOME/.gem/ruby/[ruby_base_version]).
Problems left: The user will still be able to install to the system wide gem directory /usr/lib/ruby/gems/[ruby_base_version] if they remove --user-install from /etc/gemrc
Possible remedies: 1) Leave it as is. If the user runs gem with --no-user-install then gem will still install the files to the system wide directory at /usr/lib/ruby/gems/[ruby_base_version] and will install binaries to /usr/bin This should provide a clean default and is the easiest path. This option is still better than the one we currently use where we try to install to the system wide location by default.
I think it's worth separating out the "user" and the "admin" in this argument. To install a gem system-wide, you have to do something like "sudo gem install XXX", right? This is almost always a bad idea, IMO, and people hopefully won't do it at least during the normal run of working with ruby. Personally, I don't let gem mess with anything outside my home directory.
Using --user-install with sudo will install to /root/.gem :)
And plain "gem install XXX" will not accidentally install system-wide, right?
right, putting --user-install into /etc/gemrc will put gem install foo into ~/.gem
Ruby is quite good for separating between development and deployment environments - it seems to make sense for us to do a similar thing here, seeing the Arch OS layer itself as a particular kind of deployment, where things are managed by pacman.
2) Change all ruby pacman packages to use local _gemdir='/usr/lib/ruby/vendor_ruby' This would free up /usr/lib/ruby/gems/[ruby_base_version] which we could move to /var/lib/ruby/gems/[ruby_base_version] for system wide gems installed with the gem command.
And as much as I have to admit that I'd really like to be able to install gems system-wide and use their built-in dependency management, bundler etc... I think that road probably leads to madness and death, especially when many popular gems will still be managed by pacman (and rightly so). So, I would be careful about doing this, and would suggest that we go for only letting 'gem' install things under $HOME. Anything that should go system-wide should have a PKGBUILD.
Right, by default I want it so that you wont be able to install system wide gems with gem. But I can't stop users removing --user-install from the gemrc file and installing to the system-wide directories. I have another plan brewing in regard to letting users separately install pacman installed gems and gem installed gems to the system-wide directory. I wont do it in this cleanup iteration, but next cleanup, I will add a /var/lib/gems or /var/lib/ruby/gems folder specifically so that when running sudo gem install without --user-install, it will install to /var/lib/.. and gems installed via pacman will install to /usr/lib/ruby/gems. This will cleanly seperate pacman installed ruby gems to /usr and sudo-gem without --user-install installed gems to /var. This will bring ruby and rubygems into fhs compliance, at least afaik :) Doing this will require modifying the GEM_PATH and all PKGBUILDs that install ruby gems. This wont break the PKGBUILDs just make them uncompliant with our standards, instead it will just install to /var if you don't change them which will be in GEM_PATH. BTW, GEM_PATH is the location where rubygems searchs for gems. I have already talked with upstream rubygems devs and they seem to agree that this is a good plan :)
I'm currently leaning towards 1 since it will be a simple solution with no changes needed to existing ruby-* packages.
Yep.
I'm certain I will go with 1 this cleanup and will implement the above on the next cleanup.
Of course, suggesting RVM for user-installed gems(ets) is probably also a good idea.
For rails development, I can't recommend rvm enough, you can manage multiple gemsets with multiple ruby versions, and have it automatically load project specific .rvmrc files when you cd into those directories. This is probably one of the nicest language management tools for a programming language I have encountered so I would recommend it :) Ofc, rvm might be overkill if you're just doing development on small projects, or if you just don't need a clean separation of multiple ruby environments.
Pete.
PS. A while ago I was getting very frustrated that the version of rubygems bundled with ruby itself was so out-of-date so quickly, but 'gem update --system' is a very bad idea in combination with pacman. So, I spent a while hacking together this package, which installs the latest version of rubygems itself alongside ruby, system-wide:
https://aur.archlinux.org/packages.php?ID=50346
It's hacky, but it works :-) I wish there was a cleaner way.
I was also thinking about this, and I might split rubygems off into a seperate package and have ruby optdepend or depend on it so I can update the package separately from ruby.
On Monday 13 Feb 2012 20:47:01 Thomas Dziedzic wrote:
/etc/gemrc - contains "gem: --user-install" to install user installed gems with gem to $HOME/.gem/gems
I didn't know about --user-install, but I just set GEM_HOME (actually, I use RVM). Can what you want be done by globally setting GEM_HOME to be based off of $HOME? Or is --user-install the preferred way?
I did look into $GEM_HOME but I thought this was inadequate since running sudo gem install foo would still install foo to the system-wide gem directory. Having --user-install in /etc/gemrc forces it to install to /root/.gem/gems instead.
Great, I should have read up on --user-install then. Sounds great.
And as much as I have to admit that I'd really like to be able to install gems system-wide and use their built-in dependency management, bundler etc... I think that road probably leads to madness and death, especially when many popular gems will still be managed by pacman (and rightly so). So, I would be careful about doing this, and would suggest that we go for only letting 'gem' install things under $HOME. Anything that should go system-wide should have a PKGBUILD.
Right, by default I want it so that you wont be able to install system wide gems with gem. But I can't stop users removing --user-install from the gemrc file and installing to the system-wide directories.
Perhaps we have an argument for proposing an option like --user-install that doesn't allow users to disable it then.
I have another plan brewing in regard to letting users separately install pacman installed gems and gem installed gems to the system-wide directory. I wont do it in this cleanup iteration, but next cleanup, I will add a /var/lib/gems or /var/lib/ruby/gems folder specifically so that when running sudo gem install without --user-install, it will install to /var/lib/.. and gems installed via pacman will install to /usr/lib/ruby/gems. This will cleanly seperate pacman installed ruby gems to /usr and sudo-gem without --user-install installed gems to /var. This will bring ruby and rubygems into fhs compliance, at least afaik :)
Nice. Just out of interest, what RUBYLIB order precedence would you use, to deal with the case when some similar things (say different versions of the same gem) were installed in /usr/lib/ruby and /var/lib/gems at the same time?
Doing this will require modifying the GEM_PATH and all PKGBUILDs that install ruby gems. This wont break the PKGBUILDs just make them uncompliant with our standards, instead it will just install to /var if you don't change them which will be in GEM_PATH. BTW, GEM_PATH is the location where rubygems searchs for gems.
I have already talked with upstream rubygems devs and they seem to agree that this is a good plan :)
I'm certain I will go with 1 this cleanup and will implement the above on the next cleanup.
All great.
Of course, suggesting RVM for user-installed gems(ets) is probably also a good idea.
For rails development, I can't recommend rvm enough, you can manage multiple gemsets with multiple ruby versions, and have it automatically load project specific .rvmrc files when you cd into those directories. This is probably one of the nicest language management tools for a programming language I have encountered so I would recommend it :)
Ofc, rvm might be overkill if you're just doing development on small projects, or if you just don't need a clean separation of multiple ruby environments.
Agreed. It really is a breeze to use. I don't do much rails, but just use clean gemsets along with bundler, most of the time. It helps with predicting what's going to happen when I want to run the code on another machine.
PS. A while ago I was getting very frustrated that the version of rubygems bundled with ruby itself was so out-of-date so quickly, but 'gem update --system' is a very bad idea in combination with pacman. So, I spent a while hacking together this package, which installs the latest version of rubygems itself alongside ruby, system-wide:
https://aur.archlinux.org/packages.php?ID=50346
It's hacky, but it works :-) I wish there was a cleaner way.
I was also thinking about this, and I might split rubygems off into a seperate package and have ruby optdepend or depend on it so I can update the package separately from ruby.
Hell yeah. Please do. I assume you're aware of this: https://bugs.archlinux.org/task/17611 Pete.
On Tue, Feb 14, 2012 at 3:04 AM, Peter Lewis
On Monday 13 Feb 2012 20:47:01 Thomas Dziedzic wrote:
/etc/gemrc - contains "gem: --user-install" to install user installed gems with gem to $HOME/.gem/gems
I didn't know about --user-install, but I just set GEM_HOME (actually, I use RVM). Can what you want be done by globally setting GEM_HOME to be based off of $HOME? Or is --user-install the preferred way?
I did look into $GEM_HOME but I thought this was inadequate since running sudo gem install foo would still install foo to the system-wide gem directory. Having --user-install in /etc/gemrc forces it to install to /root/.gem/gems instead.
Great, I should have read up on --user-install then. Sounds great.
And as much as I have to admit that I'd really like to be able to install gems system-wide and use their built-in dependency management, bundler etc... I think that road probably leads to madness and death, especially when many popular gems will still be managed by pacman (and rightly so). So, I would be careful about doing this, and would suggest that we go for only letting 'gem' install things under $HOME. Anything that should go system-wide should have a PKGBUILD.
Right, by default I want it so that you wont be able to install system wide gems with gem. But I can't stop users removing --user-install from the gemrc file and installing to the system-wide directories.
Perhaps we have an argument for proposing an option like --user-install that doesn't allow users to disable it then.
I think just making --user-install the default will be fine. I don't want to stop people from doing this completely since there will always be around it.
I have another plan brewing in regard to letting users separately install pacman installed gems and gem installed gems to the system-wide directory. I wont do it in this cleanup iteration, but next cleanup, I will add a /var/lib/gems or /var/lib/ruby/gems folder specifically so that when running sudo gem install without --user-install, it will install to /var/lib/.. and gems installed via pacman will install to /usr/lib/ruby/gems. This will cleanly seperate pacman installed ruby gems to /usr and sudo-gem without --user-install installed gems to /var. This will bring ruby and rubygems into fhs compliance, at least afaik :)
Nice. Just out of interest, what RUBYLIB order precedence would you use, to deal with the case when some similar things (say different versions of the same gem) were installed in /usr/lib/ruby and /var/lib/gems at the same time?
The order will be $HOME/.gem/ruby, /var/lib/gems, /usr/lib/ruby/gems Just to reiterate, the var path will be for system wide gems installed with gem. The usr path will be for gems installed with a PKGBUILD. Upstream seems to be fine with this order also.
Doing this will require modifying the GEM_PATH and all PKGBUILDs that install ruby gems. This wont break the PKGBUILDs just make them uncompliant with our standards, instead it will just install to /var if you don't change them which will be in GEM_PATH. BTW, GEM_PATH is the location where rubygems searchs for gems.
I have already talked with upstream rubygems devs and they seem to agree that this is a good plan :)
I'm certain I will go with 1 this cleanup and will implement the above on the next cleanup.
All great.
Of course, suggesting RVM for user-installed gems(ets) is probably also a good idea.
For rails development, I can't recommend rvm enough, you can manage multiple gemsets with multiple ruby versions, and have it automatically load project specific .rvmrc files when you cd into those directories. This is probably one of the nicest language management tools for a programming language I have encountered so I would recommend it :)
Ofc, rvm might be overkill if you're just doing development on small projects, or if you just don't need a clean separation of multiple ruby environments.
Agreed. It really is a breeze to use. I don't do much rails, but just use clean gemsets along with bundler, most of the time. It helps with predicting what's going to happen when I want to run the code on another machine.
PS. A while ago I was getting very frustrated that the version of rubygems bundled with ruby itself was so out-of-date so quickly, but 'gem update --system' is a very bad idea in combination with pacman. So, I spent a while hacking together this package, which installs the latest version of rubygems itself alongside ruby, system-wide:
https://aur.archlinux.org/packages.php?ID=50346
It's hacky, but it works :-) I wish there was a cleaner way.
I was also thinking about this, and I might split rubygems off into a seperate package and have ruby optdepend or depend on it so I can update the package separately from ruby.
Hell yeah. Please do. I assume you're aware of this:
I wasn't aware of this bug report, but I was aware of the recently added configure flag: --disable-rubygems disable rubygems by default
Pete.
On Tuesday 14 Feb 2012 01:53:01 Peter Lewis wrote:
I think it's worth separating out the "user" and the "admin" in this argument. To install a gem system-wide, you have to do something like "sudo gem install XXX", right? This is almost always a bad idea, IMO, and people hopefully won't do it at least during the normal run of working with ruby. Personally, I don't let gem mess with anything outside my home directory.
I'm worried. I use "sudo gem install" to install gems system-wide. It's not practical to use Arch packages for this instead, because I can't rely on a given gem always being packaged for Arch. I also don't really see the point of packaging a package from another packaging system which already happily co- exists with pacman. I have no issue using gem to administer system gems, since they always stay in /usr/lib/ruby and don't interfere with the rest of the system (except for /usr/bin, admittedly). Can you clarify if you're planning to make it impossible to use gem to install system gems? I *really* want to avoid using pacman for system gems and gem elsewhere. It makes no sense to me. I'd rather use gem for all gems. Maybe a quick summary of the plans so far? Paul
On Wed, Feb 15, 2012 at 3:58 AM, Paul Gideon Dann
On Tuesday 14 Feb 2012 01:53:01 Peter Lewis wrote:
I think it's worth separating out the "user" and the "admin" in this argument. To install a gem system-wide, you have to do something like "sudo gem install XXX", right? This is almost always a bad idea, IMO, and people hopefully won't do it at least during the normal run of working with ruby. Personally, I don't let gem mess with anything outside my home directory.
I'm worried. I use "sudo gem install" to install gems system-wide. It's not practical to use Arch packages for this instead, because I can't rely on a given gem always being packaged for Arch. I also don't really see the point of packaging a package from another packaging system which already happily co- exists with pacman. I have no issue using gem to administer system gems, since they always stay in /usr/lib/ruby and don't interfere with the rest of the system (except for /usr/bin, admittedly).
Some people choose to use pacman packaged gems because it handles deps better and patches could get included to fix some issues. I will only be implementing the changes in the planned layout only this cleanup. That means I wont be moving system-wide installation to /var/.. yet Don't worry, I wont forget about users that use gem to install to system wide locations. As already mentioned, I will make the default gem install to $HOME/.gem/ruby and if you so choose, you can edit your gemrc to install to the system wide location (/usr/lib.. for now).
Can you clarify if you're planning to make it impossible to use gem to install system gems? I *really* want to avoid using pacman for system gems and gem elsewhere. It makes no sense to me. I'd rather use gem for all gems.
I wont be forcing anyone to use pacman instead of gems, now or in the future.
Maybe a quick summary of the plans so far?
Paul
Summary of the changes in this cleanup: /usr/lib/ruby/site_ruby - This directory is for user specific installation and should never be touched by the package manager. /usr/lib/ruby/vendor_ruby - ruby packages installed with pacman which aren't gems go here $HOME/.gem/ruby/[ruby_base_version] - default target when running gem install foo because --user-install is now in the gemrc file /etc/gemrc - contains "gem: --user-install" to install user installed gems with gem to $HOME/.gem/gems If the user chooses to install gems using gem, they will have to add the bin directory to the $PATH: export PATH="$PATH:$(ruby -rubygems -e 'puts Gem.user_dir')/bin". System wide installation of gems by default will be disabled. If you want system wide gems, either run gem with the --no-user-install flag like "sudo gem install --no-user-install foo" You can also install to the system wide location by removing --user-install from /etc/gemrc
On Wednesday 15 Feb 2012 08:59:10 Thomas Dziedzic wrote:
/usr/lib/ruby/site_ruby - This directory is for user specific installation and should never be touched by the package manager. /usr/lib/ruby/vendor_ruby - ruby packages installed with pacman which aren't gems go here $HOME/.gem/ruby/[ruby_base_version] - default target when running gem install foo because --user-install is now in the gemrc file /etc/gemrc - contains "gem: --user-install" to install user installed gems with gem to $HOME/.gem/gems
If the user chooses to install gems using gem, they will have to add the bin directory to the $PATH: export PATH="$PATH:$(ruby -rubygems -e 'puts Gem.user_dir')/bin".
System wide installation of gems by default will be disabled. If you want system wide gems, either run gem with the --no-user-install flag like "sudo gem install --no-user-install foo" You can also install to the system wide location by removing --user-install from /etc/gemrc
This all sounds mostly fine. I assume that "sudo gem install" will simply install the gem using root-owned files in the home directory, right? Paul
On Wed, Feb 15, 2012 at 9:06 AM, Paul Gideon Dann
On Wednesday 15 Feb 2012 08:59:10 Thomas Dziedzic wrote:
/usr/lib/ruby/site_ruby - This directory is for user specific installation and should never be touched by the package manager. /usr/lib/ruby/vendor_ruby - ruby packages installed with pacman which aren't gems go here $HOME/.gem/ruby/[ruby_base_version] - default target when running gem install foo because --user-install is now in the gemrc file /etc/gemrc - contains "gem: --user-install" to install user installed gems with gem to $HOME/.gem/gems
If the user chooses to install gems using gem, they will have to add the bin directory to the $PATH: export PATH="$PATH:$(ruby -rubygems -e 'puts Gem.user_dir')/bin".
System wide installation of gems by default will be disabled. If you want system wide gems, either run gem with the --no-user-install flag like "sudo gem install --no-user-install foo" You can also install to the system wide location by removing --user-install from /etc/gemrc
This all sounds mostly fine. I assume that "sudo gem install" will simply install the gem using root-owned files in the home directory, right?
Paul
Correct, by default sudo gem install foo would install to /root/.gem/ruby which isn't system wide. If you really want system wide installs, either run sudo gem with --no-user-install or remove --user-install from /etc/gemrc
participants (3)
-
Paul Gideon Dann
-
Peter Lewis
-
Thomas Dziedzic