Hello!
After package manager which can server both official repos & AUR,there is another thing which I consider before returning back to Arch - filesystem.
On FreeBSD I now use zfs running on two 1TB disks in mirror (raid-1) mode.
I'd like to have similar setup on Arch, but without using lvm2+raid-1 (if possible), so I did research a bit about zfs on Linux and it looks that the 'real' name for it is Btrfs.
Afaics, btrfs will soon get fsck tool for repairing the fs, but I wonder what's the plan for btrfs support in Arch, iow, when it can become 'official' ?
Sincerely, Gour
Hi Gour,
Have you heard about the in-kernel ZFS for Linux? (http://zfsonlinux.org) project? I think there are some AUR packages for it.
Thanks. Okky Hendriansyah
-----Original Message----- From: Gour-Gadadhara Dasa gour@atmarama.net Sender: arch-general-bounces@archlinux.orgDate: Mon, 22 Aug 2011 08:17:49 To: arch-general@archlinux.org Reply-To: General Discussion about Arch Linux arch-general@archlinux.org Subject: [arch-general] btrfs & Arch
Hello!
After package manager which can server both official repos & AUR,there is another thing which I consider before returning back to Arch - filesystem.
On FreeBSD I now use zfs running on two 1TB disks in mirror (raid-1) mode.
I'd like to have similar setup on Arch, but without using lvm2+raid-1 (if possible), so I did research a bit about zfs on Linux and it looks that the 'real' name for it is Btrfs.
Afaics, btrfs will soon get fsck tool for repairing the fs, but I wonder what's the plan for btrfs support in Arch, iow, when it can become 'official' ?
Sincerely, Gour
On Mon, 22 Aug 2011 06:24:43 +0000 "Okky Hendriansyah" mawcikurl@gmail.com wrote:
Have you heard about the in-kernel ZFS for Linux?
I did, but got feeling that btrfs might be better option for the Linux.
Sincerely, Gour
Am 22.08.2011 08:17, schrieb Gour-Gadadhara Dasa:
I'd like to have similar setup on Arch, but without using lvm2+raid-1 (if possible),
Using LVM and software RAID is the right thing to do here. Any particular reason why you don't want it?
so I did research a bit about zfs on Linux and it looks that the 'real' name for it is Btrfs.
btrfs is not ZFS. and btrfs is still experimental, and I think there are some people here who had bad experience with it.
On Mon, 22 Aug 2011 09:38:52 +0200 Thomas Bächler thomas@archlinux.org wrote:
Using LVM and software RAID is the right thing to do here. Any particular reason why you don't want it?
To have one layer less.
btrfs is not ZFS. and btrfs is still experimental, and I think there are some people here who had bad experience with it.
That's good to know. Thanks.
Sincerely, Gour
On Mon, Aug 22, 2011 at 5:19 AM, Gour-Gadadhara Dasa gour@atmarama.net wrote:
On Mon, 22 Aug 2011 09:38:52 +0200 Thomas Bächler thomas@archlinux.org wrote:
Using LVM and software RAID is the right thing to do here. Any particular reason why you don't want it?
To have one layer less.
btrfs is not ZFS. and btrfs is still experimental, and I think there are some people here who had bad experience with it.
That's good to know. Thanks.
i can more or less echo what Jan said -- running here since circa .30 as well. i totally fsck'd (pun intended :-) my FS once, and have needed to zero the log tree twice in that time, although i likely punish the FS a bit more than average user likely ... the time i hosed it i was doing some funky --bind mounts *through* and *directly to* the "special" subvolume dirs.
if you want to use it you'll probably just want to keep close eye on development and be conservative about chosen options (eg. bootloaders cant handle compression/etc), but IME at least it's been pretty solid all things considered :-)
... if you need an external backup solution it works great as a fast/efficient differential system, whereby each "backup" is an really an in-place rsync to a new snapshot. there are folks working on a means to "remote-send" the FS to other destinations, now sure the exact details on that one.
... and the fsck appears to be literally days away ... Chris just mentioned it again a day or two ago before going on vacation, though it's seemed close for some time now. in short, it's definitely doable, and it's crammed with a bunch of interesting features, but you should at this point still be prepared to recover and/or lose it all (but to be fair, in a couple years on the list i haven't seen many FS'es totally nuked)
Am 22.08.2011 18:35 schrieb "C Anthony Risinger" anthony@xtfx.me:
On Mon, Aug 22, 2011 at 5:19 AM, Gour-Gadadhara Dasa gour@atmarama.net
wrote:
On Mon, 22 Aug 2011 09:38:52 +0200 Thomas Bächler thomas@archlinux.org wrote:
Using LVM and software RAID is the right thing to do here. Any particular reason why you don't want it?
To have one layer less.
btrfs is not ZFS. and btrfs is still experimental, and I think there are some people here who had bad experience with it.
That's good to know. Thanks.
i can more or less echo what Jan said -- running here since circa .30 as well. i totally fsck'd (pun intended :-) my FS once, and have needed to zero the log tree twice in that time, although i likely punish the FS a bit more than average user likely ... the time i hosed it i was doing some funky --bind mounts *through* and *directly to* the "special" subvolume dirs.
if you want to use it you'll probably just want to keep close eye on development and be conservative about chosen options (eg. bootloaders cant handle compression/etc), but IME at least it's been pretty solid all things considered :-)
... if you need an external backup solution it works great as a fast/efficient differential system, whereby each "backup" is an really an in-place rsync to a new snapshot. there are folks working on a means to "remote-send" the FS to other destinations, now sure the exact details on that one.
... and the fsck appears to be literally days away ... Chris just mentioned it again a day or two ago before going on vacation, though it's seemed close for some time now. in short, it's definitely doable, and it's crammed with a bunch of interesting features, but you should at this point still be prepared to recover and/or lose it all (but to be fair, in a couple years on the list i haven't seen many FS'es totally nuked)
--
C Anthony
Hi
I was about to use btrfs for my new SSD but I couldn't find any reason why I shouldn't stay with my normal LVM and ext4 setup. Is there any particular reason why everyone is so crazy about that new fs?
Schneida
On Mon, Aug 22, 2011 at 1:39 PM, Simon Schneider schneida.simon@gmail.com wrote:
Hi
I was about to use btrfs for my new SSD but I couldn't find any reason why I shouldn't stay with my normal LVM and ext4 setup. Is there any particular reason why everyone is so crazy about that new fs?
for me at least i like how simple it is to try stuff ... one of my favorite features is throwaway snapshots for testing various *whatever*. i can setup a chroot and instantly clone it and throw away just as fast; IIRC `mkchrootpkg` does this by default it it detects btrfs.
LZO compression on my eee s101 netbook (32GB "aftermarket" Runcore SSD) has made a *definite* difference in system performance.
... and also a simple rsync (inplace) script can give you fast/efficient "folding"/differential backups with very little work (and this will work even better once offline [or online i guess] de-dup is rocking)
i also use btrfs *heavily* in conjunction with namespace containers (eg, lxc-tools and friends) -- btrfs is the perfect companion. i can create hundreds of development/staging/production containers in seconds (if i actually needed them that fast ;-). it's the realization of namespaces at FS level.
i dont know much about LVM or soft-raid setups, and im sure much of the above can be achieved with --bind mounts, layers of this-and-that ... but btrfs is simple and effective. lastly -- and this is maybe a growing trend -- LVM works at the block level whereas btrfs is object level; btrfs can have object policies (eg RAID) with some objects partially in one state or another (eg compression), intelligent rebuilds (doesn't have to behave like a generic/dumb block device) ... this means things like RAID1 only EVER have 2 copies of an object ... even if there are 15 disks each object will only have 2 complete copies so space can be utilized more effectively ... this also helps to make full use of arrays with mismatched disk sizes/capabilities. <---(these last bits were discussed on the list recently, and i may be stating it incorrectly. by all means, correct me if necessary)
On Mon, Aug 22, 2011 at 2:17 PM, C Anthony Risinger anthony@xtfx.me wrote:
On Mon, Aug 22, 2011 at 1:39 PM, Simon Schneider schneida.simon@gmail.com wrote:
Hi
I was about to use btrfs for my new SSD but I couldn't find any reason why I shouldn't stay with my normal LVM and ext4 setup. Is there any particular reason why everyone is so crazy about that new fs?
for me at least i like how simple it is to try stuff ... one of my favorite features is throwaway snapshots for testing various *whatever*. i can setup a chroot and instantly clone it and throw away just as fast; IIRC `mkchrootpkg` does this by default it it detects btrfs.
LZO compression on my eee s101 netbook (32GB "aftermarket" Runcore SSD) has made a *definite* difference in system performance.
... and also a simple rsync (inplace) script can give you fast/efficient "folding"/differential backups with very little work (and this will work even better once offline [or online i guess] de-dup is rocking)
i also use btrfs *heavily* in conjunction with namespace containers (eg, lxc-tools and friends) -- btrfs is the perfect companion. i can create hundreds of development/staging/production containers in seconds (if i actually needed them that fast ;-). it's the realization of namespaces at FS level.
i dont know much about LVM or soft-raid setups, and im sure much of the above can be achieved with --bind mounts, layers of this-and-that ... but btrfs is simple and effective. lastly -- and this is maybe a growing trend -- LVM works at the block level whereas btrfs is object level; btrfs can have object policies (eg RAID) with some objects partially in one state or another (eg compression), intelligent rebuilds (doesn't have to behave like a generic/dumb block device) ... this means things like RAID1 only EVER have 2 copies of an object ... even if there are 15 disks each object will only have 2 complete copies so space can be utilized more effectively ... this also helps to make full use of arrays with mismatched disk sizes/capabilities. <---(these last bits were discussed on the list recently, and i may be stating it incorrectly. by all means, correct me if necessary)
while i haven't personally (er... knowingly ;-) experienced any problems, it's appropriate to mention that recently there have been some reports of silent data-corruption in some setups. i dont think the exact cause is known yet ... but it also sounds like the new btrfsck will tackle much of it.
... tbh, unless you want to get a feel for it, are ready to troubleshoot, or have a specific use-case -- and have some other backup systems in place -- you'll probably just want to roll with "the usual" for now.
On Mon, 22 Aug 2011 14:25:52 -0500 C Anthony Risinger anthony@xtfx.me wrote:
... tbh, unless you want to get a feel for it, are ready to troubleshoot, or have a specific use-case -- and have some other backup systems in place -- you'll probably just want to roll with "the usual" for now.
Let me say that I'm a bit spoiled with elegant setup I've on FreeBSD with ZFS and would like to mimic it on Linux with btrfs.
Machine is connected to UPS and I've tape backup.
However, I'm still not sure how the setup should be for 2 disks in mirror?
With ZFS, I've everything; iow. root, boot & swap in single pool which is then mirrored to the 2nd disk which is part of the same pool.
With btrfs, I understand I can avoid using separate /boot and have root on btrfs, add two disks as raid-1 under btrfs partition, but wonder whato do in regard to swap considering btrfs does not provide swap space?
Does it mean that I'd have to create separate swap partitions on each disk and put them into raid-1 array with mdadm and then put the rest of the disk(s) as btrfs partion in mirror?
Sincerely, Gour
On Wed, Aug 24, 2011 at 12:28 AM, Gour-Gadadhara Dasa gour@atmarama.net wrote:
On Mon, 22 Aug 2011 14:25:52 -0500 C Anthony Risinger anthony@xtfx.me wrote:
... tbh, unless you want to get a feel for it, are ready to troubleshoot, or have a specific use-case -- and have some other backup systems in place -- you'll probably just want to roll with "the usual" for now.
Let me say that I'm a bit spoiled with elegant setup I've on FreeBSD with ZFS and would like to mimic it on Linux with btrfs.
Machine is connected to UPS and I've tape backup.
However, I'm still not sure how the setup should be for 2 disks in mirror?
With ZFS, I've everything; iow. root, boot & swap in single pool which is then mirrored to the 2nd disk which is part of the same pool.
With btrfs, I understand I can avoid using separate /boot and have root on btrfs, add two disks as raid-1 under btrfs partition, but wonder whato do in regard to swap considering btrfs does not provide swap space?
Does it mean that I'd have to create separate swap partitions on each disk and put them into raid-1 array with mdadm and then put the rest of the disk(s) as btrfs partion in mirror?
unfortunately you can only do /boot on btrfs if you use a single disk due to bootloader limitations:
https://wiki.archlinux.org/index.php/Installing_on_Btrfs_root
... details most of the procedure, but it needs to be updated because until bootloaders and utilities can handle it better, the result is a PITA to work with ... newer btrfs features like compression render your system unbootable unless you manually decompress the kernel image, there is no partition table so you can't modify the disk layout after the fact, etc etc. page needs a warning or something.
swap has been disabled in btrfs because it will corrupt it ... btrfs move blocks around constantly while performing COW and swap file/partition expects stable address block IIRC -- which btrfs does not guarantee or even attempt -- so you end up corrupting the FS *bad*. using a loopback device + swapfile *should* work, but i don't know of anyone actually doing it yet :-)
so yeah, you'd need something like `mdadm` to handle the RAID bits. eventually i hope to get `mkinitcpio-btrfs` updated to do fancy stuff like a 2-stage kexec boot so you CAN boot from a btrfs array ... but idk when that will be :-) ... mostly working ATM but time time time ... there's never enough.
after all that, it might be simpler to just use ext4 since your running softraid anyway ... unless you want to try stuff like compression and snapshots.
On Wed, 24 Aug 2011 12:44:10 -0500 C Anthony Risinger anthony@xtfx.me wrote:
unfortunately you can only do /boot on btrfs if you use a single disk due to bootloader limitations:
Thank you for clarification...I was not really understanding the limitation.
swap has been disabled in btrfs because it will corrupt it ...
That's not such an issue. I may just create swap paritions...
so yeah, you'd need something like `mdadm` to handle the RAID bits.
OK. Thanks.
after all that, it might be simpler to just use ext4 since your running softraid anyway ... unless you want to try stuff like compression and snapshots.
I'd like to try snapshots and wonder how does compression effect performance when using e.g. lzo, iow, it is recommended for desktop machine?
Sincerely, Gour
On Wed, 24 Aug 2011 12:44:10 -0500 C Anthony Risinger anthony@xtfx.me wrote:
unfortunately you can only do /boot on btrfs if you use a single disk due to bootloader limitations:
Is it only bootloader limitation considering that wiki page says: "This setup does not support btrfs RAID because a separate boot partition/device is needed, ie. only one drive allowed. Pending releases of mkinitcpio-btrfs will attempt to provide solutions." ?
Sincerely, Gour
On Thu, Aug 25, 2011 at 2:19 AM, Gour-Gadadhara Dasa gour@atmarama.net wrote:
On Wed, 24 Aug 2011 12:44:10 -0500 C Anthony Risinger anthony@xtfx.me wrote:
unfortunately you can only do /boot on btrfs if you use a single disk due to bootloader limitations:
Is it only bootloader limitation considering that wiki page says: "This setup does not support btrfs RAID because a separate boot partition/device is needed, ie. only one drive allowed. Pending releases of mkinitcpio-btrfs will attempt to provide solutions." ?
im not sure im understanding correctly ... mkinitcpio-btrfs is just an initcpio hook performing a handful of trixy things to "rollback in time", ie. boot an older snapshot. everything it does is orthogonal to the /boot problem ... though i had some experimentations using a two-stage system that booted a minimal kernel far enough to load the real kernel off the btrfs array -- along with other activity this could "enable" /boot on a btrfs array -- not even tested yet though.
fundamentally syslinux and friends (AFAIK) are not capable of reading a disk if part of an array. syslinux can't even look inside a subvolume ... hence kernel rollbacks are not straightforward.
you *can* however have / on btrfs, *and* an array ... so long as your /boot is still using `md` + <other FS>. maybe btrfs could even be that FS ... not sure. for this to work you just need the `btrfs` hook included with Arch, or you need to pass all the devices in the array as a mount option (eg. via `rootflags` on kernel bootline)
2011/8/25 C Anthony Risinger anthony@xtfx.me
On Thu, Aug 25, 2011 at 2:19 AM, Gour-Gadadhara Dasa gour@atmarama.net wrote:
On Wed, 24 Aug 2011 12:44:10 -0500 C Anthony Risinger anthony@xtfx.me wrote:
unfortunately you can only do /boot on btrfs if you use a single disk due to bootloader limitations:
Is it only bootloader limitation considering that wiki page says: "This
setup
does not support btrfs RAID because a separate boot partition/device is
needed,
ie. only one drive allowed. Pending releases of mkinitcpio-btrfs will
attempt
to provide solutions." ?
im not sure im understanding correctly ... mkinitcpio-btrfs is just an initcpio hook performing a handful of trixy things to "rollback in time", ie. boot an older snapshot. everything it does is orthogonal to the /boot problem ... though i had some experimentations using a two-stage system that booted a minimal kernel far enough to load the real kernel off the btrfs array -- along with other activity this could "enable" /boot on a btrfs array -- not even tested yet though.
fundamentally syslinux and friends (AFAIK) are not capable of reading a disk if part of an array. syslinux can't even look inside a subvolume ... hence kernel rollbacks are not straightforward.
you *can* however have / on btrfs, *and* an array ... so long as your /boot is still using `md` + <other FS>. maybe btrfs could even be that FS ... not sure. for this to work you just need the `btrfs` hook included with Arch, or you need to pass all the devices in the array as a mount option (eg. via `rootflags` on kernel bootline)
--
C Anthony
I got even more curious about btrfs after following this discussion and installed it on my primary laptop (I planned a reinstall anyway to get rid of windows). Installation worked perfectly with Archboot, setting up two subvolumes, both compressed and everything worked perfectly for about a week: My DVR stick made the computer freeze (this happens like twice a year) and therfore I had to hard reset the machine with the effect that it rendered the filesystem unusable for the linux kernel. Grub could still read the kernel image and booted it just fine, but as soon as the kernel tried to mount the filesystem it crashed. I tried to recover with a live cd, but because I had no internet access at that time, and man pages weren't very helpful either I eventuelly just reinstalled Arch using the familiar lvm2 + ext4 setup.
Btrfs is great, don't get me wrong, it really is fast and snapshots and all that worked really great, but it is, sadly, a little bit too unstable to use it anywhere than on a test machine. Even if you have UPS and all kind of measurements to prevent such power failures and hard resets, your computer still might hang at some point and then you'll really are in big troubles.
On Mon, 29 Aug 2011 21:22:37 +0200 Simon Schneider schneida.simon@gmail.com wrote:
Btrfs is great, don't get me wrong, it really is fast and snapshots and all that worked really great, but it is, sadly, a little bit too unstable to use it anywhere than on a test machine. Even if you have UPS and all kind of measurements to prevent such power failures and hard resets, your computer still might hang at some point and then you'll really are in big troubles.
Thanks a lot for sharing very valuable information. Considering that I want to return to Linux asap, it seems it's better not to gamble and install using ext4+lvm2+raid-1 and then, when btrfs becomes more stable, detach one disk from ext4 raid-1 array, format as btrfs, move the OS & data to it, dismantle array and add 2nd disk to the btrfs raid-1.
Sincerely, Gour
On 08/29/2011 03:22 PM, Simon Schneider wrote:
I got even more curious about btrfs after following this discussion and installed it on my primary laptop (I planned a reinstall anyway to get rid of windows). Installation worked perfectly with Archboot, setting up two subvolumes, both compressed and everything worked perfectly for about a week: My DVR stick made the computer freeze (this happens like twice a year) and therfore I had to hard reset the machine with the effect that it rendered the filesystem unusable for the linux kernel. Grub could still read the kernel image and booted it just fine, but as soon as the kernel tried to mount the filesystem it crashed. I tried to recover with a live cd, but because I had no internet access at that time, and man pages weren't very helpful either I eventuelly just reinstalled Arch using the familiar lvm2 + ext4 setup. Btrfs is great, don't get me wrong, it really is fast and snapshots and all that worked really great, but it is, sadly, a little bit too unstable to use it anywhere than on a test machine. Even if you have UPS and all kind of measurements to prevent such power failures and hard resets, your computer still might hang at some point and then you'll really are in big troubles.
I had the same experience with a new Arch install. On shutdown the computer hung, and after a hard reset the file system was trashed with no way to recover. I went back to ext4. While it ran it was fast though, and I look forward to using it again, but only after an fsck utility becomes available.
Jon
On Mon, 22 Aug 2011 11:34:46 -0500 C Anthony Risinger anthony@xtfx.me wrote:
if you want to use it you'll probably just want to keep close eye on development and be conservative about chosen options (eg. bootloaders cant handle compression/etc), but IME at least it's been pretty solid all things considered :-)
Nice.
My desktop has UPS, so I hope I can avoid one kind of errors due to improper shutdown.
... if you need an external backup solution it works great as a fast/efficient differential system, whereby each "backup" is an really an in-place rsync to a new snapshot. there are folks working on a means to "remote-send" the FS to other destinations, now sure the exact details on that one.
No, for backup I use Amanda and LTO (2) tapes and the btrfs would serve my desktop.
... and the fsck appears to be literally days away ... Chris just mentioned it again a day or two ago before going on vacation, though it's seemed close for some time now. in short, it's definitely doable, and it's crammed with a bunch of interesting features, but you should at this point still be prepared to recover and/or lose it all (but to be fair, in a couple years on the list i haven't seen many FS'es totally nuked)
For now I'll experiment with my old laptop, but I had some GPT/syslinux related issues with install...let me resolve those first.
Sincerely, Gour
On Mon, Aug 22, 2011 at 8:17 AM, Gour-Gadadhara Dasa gour@atmarama.net wrote:
Afaics, btrfs will soon get fsck tool for repairing the fs, but I wonder what's the plan for btrfs support in Arch, iow, when it can become 'official' ?
BTRFS is already supported by initscripts: https://projects.archlinux.org/initscripts.git/commit/?id=ca372312062e7843ca...
and package is in core distro repository: http://www.archlinux.org/packages/core/x86_64/btrfs-progs-unstable/
This sounds "official". But as underline Thomas, btrfs is still unstable ! However, ubuntu does not seem to be afraid : https://help.ubuntu.com/community/btrfs
On Mon, 22 Aug 2011 10:43:58 +0200 Seblu seblu@seblu.net wrote:
On Mon, Aug 22, 2011 at 8:17 AM, Gour-Gadadhara Dasa gour@atmarama.net wrote:
Afaics, btrfs will soon get fsck tool for repairing the fs, but I wonder what's the plan for btrfs support in Arch, iow, when it can become 'official' ?
BTRFS is already supported by initscripts: https://projects.archlinux.org/initscripts.git/commit/?id=ca372312062e7843ca...
and package is in core distro repository: http://www.archlinux.org/packages/core/x86_64/btrfs-progs-unstable/
This sounds "official". But as underline Thomas, btrfs is still unstable ! However, ubuntu does not seem to be afraid : https://help.ubuntu.com/community/btrfs
There is enough reason to be afraid. After 4 weeks of unencumbered using, rm'ing a directory failed with the error "directory not empty". A fs check showed errors, as expected, but of course I wasn't able to repair it. Now I'm back to ext4.
On Mon, Aug 22, 2011 at 8:17 AM, Gour-Gadadhara Dasa gour@atmarama.net wrote:
Hello!
After package manager which can server both official repos & AUR,there is another thing which I consider before returning back to Arch - filesystem.
On FreeBSD I now use zfs running on two 1TB disks in mirror (raid-1) mode.
I'd like to have similar setup on Arch, but without using lvm2+raid-1 (if possible), so I did research a bit about zfs on Linux and it looks that the 'real' name for it is Btrfs.
Afaics, btrfs will soon get fsck tool for repairing the fs, but I wonder what's the plan for btrfs support in Arch, iow, when it can become 'official' ?
Sincerely, Gour
-- “In the material world, conceptions of good and bad are all mental speculations…” (Sri Caitanya Mahaprabhu)
http://atmarama.net | Hlapicina (Croatia) | GPG: 52B5C810
btrfs may still fail to mount when not unmounted cleanly in some cases. There are fsck and patches in the works to make it more tolerant, but these are not available at this time.
That said, I've been running it since 2.6.30 as my /. A bad interaction with Tux-On-Ice killed it once, but I haven't had any irreparable problems since then.
arch-general@lists.archlinux.org