[arch-general] Hacking into HAL's mount process
By default HAL adds the flush flag when USB devices are mounted... I think which is what making WRITEs slow. I want that HAL shouldn't add the flush flag. How to ? -- Nilesh Govindarajan Site & Server Administrator www.itech7.com
On 13/03/2010, Nilesh Govindarajan <lists@itech7.com> wrote:
By default HAL adds the flush flag when USB devices are mounted... I think which is what making WRITEs slow. I want that HAL shouldn't add the flush flag. How to ?
Funny..I remember the very reason for making "flush" a default mount option was because users were frantically trying to convince everyone around that it would allow for optimal performance of flash-based storage, aside from running the risk of losing data without it. -- GPG/PGP ID: B42DDCAD
By default HAL adds the flush flag wh... Funny..I remember the very reason for making "flush" a default mount
Yes, it's all placebo effect. What seems like faster transfers is really the use of write caching and not a good idea for removable media. Could leave the data and filesystem in inconsistent states if the device were accidentally removed or if power failed. On Mar 14, 2010 4:33 AM, "Ray Rashif" <schivmeister@gmail.com> wrote: On 13/03/2010, Nilesh Govindarajan <lists@itech7.com> wrote: option was because users were frantically trying to convince everyone around that it would allow for optimal performance of flash-based storage, aside from running the risk of losing data without it. -- GPG/PGP ID: B42DDCAD
On 03/14/2010 07:41 PM, Robert Howard wrote:
Yes, it's all placebo effect. What seems like faster transfers is really the use of write caching and not a good idea for removable media. Could leave the data and filesystem in inconsistent states if the device were accidentally removed or if power failed.
On Mar 14, 2010 4:33 AM, "Ray Rashif"<schivmeister@gmail.com> wrote:
By default HAL adds the flush flag wh... Funny..I remember the very reason for making "flush" a default mount
On 13/03/2010, Nilesh Govindarajan<lists@itech7.com> wrote: option was because users were frantically trying to convince everyone around that it would allow for optimal performance of flash-based storage, aside from running the risk of losing data without it.
-- GPG/PGP ID: B42DDCAD
@Ray, If the device has small buffer size, then it takes lot of time. Who want's to shell out more bucks for this simple flush thing to get a device with bigger buffer size. @Robert, Power would fail if there was no UPS. But I have one. Also, my transfers are super slow. It takes more than 15 minutes to transfer 500 megs !! Previously I used to boast among my friends for Linux being fast with USB transfers. -- Nilesh Govindarajan Site & Server Adminstrator www.itech7.com
On 03/14/2010 02:21 PM, Nilesh Govindarajan wrote:
On 03/14/2010 07:41 PM, Robert Howard wrote:
Yes, it's all placebo effect. What seems like faster transfers is really the use of write caching and not a good idea for removable media. Could leave the data and filesystem in inconsistent states if the device were accidentally removed or if power failed.
On Mar 14, 2010 4:33 AM, "Ray Rashif"<schivmeister@gmail.com> wrote:
By default HAL adds the flush flag wh... Funny..I remember the very reason for making "flush" a default mount
On 13/03/2010, Nilesh Govindarajan<lists@itech7.com> wrote: option was because users were frantically trying to convince everyone around that it would allow for optimal performance of flash-based storage, aside from running the risk of losing data without it.
-- GPG/PGP ID: B42DDCAD
@Ray, If the device has small buffer size, then it takes lot of time. Who want's to shell out more bucks for this simple flush thing to get a device with bigger buffer size.
@Robert, Power would fail if there was no UPS. But I have one.
Also, my transfers are super slow. It takes more than 15 minutes to transfer 500 megs !!
Previously I used to boast among my friends for Linux being fast with USB transfers.
I have to agree with the use of an UPS, it helps not only to avoid data loss but also hardware damage, and these days you can get a cheap UPS that can supply power for a reasonable amount of time. On the device being accidentally removed, either way you can have data loss, also people should really unmount and check the device is idle before removing it, otherwise bad things can happen. Most devices have an activity indicator and users should have the good sense to look at that indicator (when it exists) to check for activity before removing the device, even when properly unmounting a device ... although I guess some users will never learn and will complain when bad thing happen because of a problem that is between the keyboard and the chair. Caching may also help if you perform several changes (writes and reads) in the device before committing all changes, as in deleting many files or writing many files.
On Sun, Mar 14, 2010 at 07:51:28PM +0530, Nilesh Govindarajan wrote:
@Ray, If the device has small buffer size, then it takes lot of time. Who want's to shell out more bucks for this simple flush thing to get a device with bigger buffer size.
@Robert, Power would fail if there was no UPS. But I have one.
Also, my transfers are super slow. It takes more than 15 minutes to transfer 500 megs !!
Previously I used to boast among my friends for Linux being fast with USB transfers.
Just put the device in /etc/fstab with whatever options you want, (u)mount it manually and forget about hal. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !
On 03/14/2010 08:50 PM, fons@kokkinizita.net wrote:
On Sun, Mar 14, 2010 at 07:51:28PM +0530, Nilesh Govindarajan wrote:
@Ray, If the device has small buffer size, then it takes lot of time. Who want's to shell out more bucks for this simple flush thing to get a device with bigger buffer size.
@Robert, Power would fail if there was no UPS. But I have one.
Also, my transfers are super slow. It takes more than 15 minutes to transfer 500 megs !!
Previously I used to boast among my friends for Linux being fast with USB transfers.
Just put the device in /etc/fstab with whatever options you want, (u)mount it manually and forget about hal.
Ciao,
Doesn't do good. I have multiple pen drives :) -- Nilesh Govindarajan Site & Server Adminstrator www.itech7.com
Am oder ungefähr am Sonntag, 14. März 2010, schrieb Nilesh Govindarajan:
On 03/14/2010 08:50 PM, fons@kokkinizita.net wrote:
On Sun, Mar 14, 2010 at 07:51:28PM +0530, Nilesh Govindarajan wrote:
Just put the device in /etc/fstab with whatever options you want, (u)mount it manually and forget about hal.
Ciao,
Doesn't do good. I have multiple pen drives :)
In this case you can either use labels or uuids to identify them in /etc/fstab, e.g. /dev/disk/by-label/Data /mnt/Data vfat uid=root,gid=users,showexec,user,noauto,umask=002,utf8,shortname=mixed 0 0 /dev/disk/by-uuid/55ba4890-140e-4087-8851-3c17b295c421 /mnt/Archiv ext4 defaults,user,exec,noauto 0 0 :) Ed
On Sun, Mar 14, 2010 at 05:02:47PM +0100, Edgar Kalkowski wrote:
In this case you can either use labels or uuids to identify them in /etc/fstab, e.g.
/dev/disk/by-label/Data /mnt/Data vfat uid=root,gid=users,showexec,user,noauto,umask=002,utf8,shortname=mixed 0 0 /dev/disk/by-uuid/55ba4890-140e-4087-8851-3c17b295c421 /mnt/Archiv ext4 defaults,user,exec,noauto 0 0
Don't know for the uuid, but for a label you can just write: LABEL=/Data /mnt/Data vfat uid=root,gid=users,showexec,user,noauto,umask=002,utf8,shortname=mixed 0 0 as well. -- FA O tu, che porte, correndo si ? E guerra e morte !
Am oder ungefähr am Sonntag, 14. März 2010, schrieb fons@kokkinizita.net:
On Sun, Mar 14, 2010 at 05:02:47PM +0100, Edgar Kalkowski wrote:
In this case you can either use labels or uuids to identify them in /etc/fstab, e.g.
/dev/disk/by-label/Data /mnt/Data vfat uid=root,gid=users,showexec,user,noauto,umask=002,utf8,shortname=mixed 0 0 /dev/disk/by-uuid/55ba4890-140e-4087-8851-3c17b295c421 /mnt/Archiv ext4 defaults,user,exec,noauto 0 0
Don't know for the uuid, but for a label you can just write:
LABEL=/Data /mnt/Data vfat uid=root,gid=users,showexec,user,noauto,umask=002,utf8,shortname=mixed 0 0
as well.
This works with UUID=… as well but I had some problems in the past with the KDE device notifier refusing to mount those devices with an error message like “that device is listed in /etc/fstab, refusing to mount”. This did not happen any longer after I switched to /dev/disk/by-uuid/….
Anyway, a slightly off-topic complaint I have is that my 32GB Cruzer is slow as hell to write to at just a measly 3MB/s. Sheesh. I wonder if it's the size and technical limitation or whether there might be some untested hackery. I was thinking of formatting it in Windows and changing the allocation unit to 64K, but I recall it only helped in reads. It was only formatted a number of times before on Linux. First to get rid of whatever plug-and-play firmware/software bundle it came with, and then to play around with the best partitioning scheme, after which I just had to abandon the idea because Windows didn't want to be friendly with the main partition and Macs just plain ignored the drive entirely. A layman benchmark with two other similar smaller-sized drives, and switching between fat16 and 32, provides a hypothesis that > size == < speed && > fat16 == < speed. A friend has even tried exFat but (1) Linux has only experimental read support for that and (2) its speed is as worse as the worst I am used to.
On 03/14/10 12:30, Ray Rashif wrote:
Anyway, a slightly off-topic complaint I have is that my 32GB Cruzer is slow as hell to write to at just a measly 3MB/s.
My SanDisk Sansa Clip in mass-storage mode is excessively slow (also, Linux used to have difficulty mounting it in USB 2.0 mode and either fell back to 1.1 or failed). So maybe it's SanDisk's fault.
A layman benchmark with two other similar smaller-sized drives, and switching between fat16 and 32, provides a hypothesis that> size ==< speed&& > fat16 ==< speed. A friend has even tried exFat but (1) Linux has only experimental read support for that and (2) its speed is as worse as the worst I am used to.
Try UDF filesystem. It was originally invented for data DVDs, but I read that*, these days, Linux, Windows and Mac can all read *and write* to it (maybe due to DVD+RAM?) and that it is therefore a better successor to FAT than a newfangled Microsoft format. *there was an article I can't find again, but Wikipedia says, http://en.wikipedia.org/wiki/Universal_Disk_Format#Native_OS_support read support has been around for a while, and write support since Windows Vista, Linux 2.6, Mac OS X 10.5 -Isaac
So far, after eliminating flush option, I achieved a speed of 3.9 MB/s which was previously 1-2 MB/s. If I could somehow force the async option, I think speed would increase more. Any ideas ? The async option appears true (configured) in hal-device, but doesn't get mounted with that option. -- Nilesh Govindarajan Site & Server Administrator www.itech7.com
On Sun, Mar 14, 2010 at 08:58:45PM +0530, Nilesh Govindarajan wrote:
On 03/14/2010 08:50 PM, fons@kokkinizita.net wrote:
Just put the device in /etc/fstab with whatever options you want, (u)mount it manually and forget about hal.
Doesn't do good. I have multiple pen drives :)
Same here, no problem. Label them, use the label to identify them in /etc/fstab, and have each of them mounted to its own fixed place. Ciao, -- FA O tu, che porte, correndo si ? E guerra e morte !
On 15 March 2010 00:03, <fons@kokkinizita.net> wrote:
On Sun, Mar 14, 2010 at 08:58:45PM +0530, Nilesh Govindarajan wrote:
On 03/14/2010 08:50 PM, fons@kokkinizita.net wrote:
Just put the device in /etc/fstab with whatever options you want, (u)mount it manually and forget about hal.
Doesn't do good. I have multiple pen drives :)
Same here, no problem. Label them, use the label to identify them in /etc/fstab, and have each of them mounted to its own fixed place.
Anyway, back to the main topic, you have: [schiv@v3000 hal]$ pwd /usr/share/hal [schiv@v3000 hal]$ grep -R flush . ./fdi/policy/10osvendor/20-storage-methods.fdi: <append key="volume.mount.valid_options" type="strlist">flush</append> [schiv@v3000 hal]$ Just copy the file to /etc/hal/fdi/policy/10osvendor/ and remove that line (it's part of a set of mount options for vfat). -- GPG/PGP ID: B42DDCAD
Hacked mount options by hal finally. There is not much gain after disabling flush. I was getting 1-2 MB/s before flush and now 3-4 MB/s Approximately twice. I used to get around 7-10 MB/s some time ago with my same pen drives. I'm researching more to force async option. I edited /usr/share/hal/fdi/policy/10osvendor/20-storage-methods.fdi But this is not the solution because /usr/share/hal is files of the package and may get overridden on upgrade. -- Nilesh Govindarajan Site & Server Adminstrator www.itech7.com
fons@kokkinizita.net (2010-03-14 17:03):
On Sun, Mar 14, 2010 at 08:58:45PM +0530, Nilesh Govindarajan wrote:
On 03/14/2010 08:50 PM, fons@kokkinizita.net wrote:
Just put the device in /etc/fstab with whatever options you want, (u)mount it manually and forget about hal.
Doesn't do good. I have multiple pen drives :)
Same here, no problem. Label them, use the label to identify them in /etc/fstab, and have each of them mounted to its own fixed place.
In the spirit of getting away from HAL (and the topic): - just put a custom udev rule under /etc/udev/rules.d/ (sample attached). -- -- Rogutės Sparnuotos # automount vfat on removable devices (/dev/sd? -> /media/sd?) ACTION=="add", KERNEL=="sd[a-z][1-9]", ATTRS{removable}=="1", ENV{ID_FS_TYPE}=="vfat", GROUP="storage", RUN+="/bin/mkdir -p /media/%k", RUN+="/bin/mount -t vfat -o rw,noatime,gid=storage,dmask=002,fmask=002,utf8 /dev/%k /media/%k" ACTION=="remove", KERNEL=="sd[a-z][1-9]", ENV{ID_FS_TYPE}=="vfat", RUN+="/bin/umount /media/%k", RUN+="/bin/rmdir /media/%k"
participants (8)
-
Edgar Kalkowski
-
fons＠kokkinizita.net
-
Isaac Dupree
-
Mauro Santos
-
Nilesh Govindarajan
-
Ray Rashif
-
Robert Howard
-
Rogutės Sparnuotos