[arch-general] Finally found a solution to slow USB on HAL

older
[arch-general] Totem leaves dvd...

Nilesh Govindarajan

28 Jun 2010 28 Jun '10

11:25 a.m.

I have made a patch for /usr/share/hal/fdi/20-storage-methods.fdi to force async file transfer for vfat filesystems by commenting out flush and sync as valid options from the list. I checked the thing, now I'm getting the old high speed USB transfer. Do take a look at it, and comment. -- Regards, Nilesh Govindarajan Facebook: http://www.facebook.com/nilesh.gr Twitter: http://twitter.com/nileshgr Website: http://www.itech7.com Cheap and Reliable VPS Hosting: http://j.mp/arHk5e

Attachments:

20-storage-methods.fdi.patch (text/plain — 4.6 KB)

Show replies by date

Pálffy András Gergely

28 Jun 28 Jun

4:21 p.m.

Works here too. Great, thanks. On Mon, Jun 28, 2010 at 1:25 PM, Nilesh Govindarajan wrote:

...

I have made a patch for /usr/share/hal/fdi/20-storage-methods.fdi to force async file transfer for vfat filesystems by commenting out flush and sync as valid options from the list. I checked the thing, now I'm getting the old high speed USB transfer. Do take a look at it, and comment.

-- Regards, Nilesh Govindarajan Facebook: http://www.facebook.com/nilesh.gr Twitter: http://twitter.com/nileshgr Website: http://www.itech7.com Cheap and Reliable VPS Hosting: http://j.mp/arHk5e

Denis A. Altoé Falqueto

4:33 p.m.

On Mon, Jun 28, 2010 at 1:21 PM, Pálffy András Gergely wrote:

...

Works here too. Great, thanks.

On Mon, Jun 28, 2010 at 1:25 PM, Nilesh Govindarajan wrote:

...
I have made a patch for /usr/share/hal/fdi/20-storage-methods.fdi to force async file transfer for vfat filesystems by commenting out flush and sync as valid options from the list. I checked the thing, now I'm getting the old high speed USB transfer. Do take a look at it, and comment.

Yes, but you should keep in mind that you'll spend extra time when you want to unmount your USB stick. So I prefer a "slow" transfer and a fast unmount, because usually I'm in hurry for taking off the USB drive and the unmounting visualizations are not very smart (only in KDE SC 4.4 it is really usable). -- A: Because it obfuscates the reading. Q: Why is top posting so bad? ------------------------------------------- Denis A. Altoe Falqueto -------------------------------------------

Philipp Überbacher

8:52 p.m.

Excerpts from Denis A. Altoé Falqueto's message of 2010-06-28 18:33:30 +0200:

...

On Mon, Jun 28, 2010 at 1:21 PM, Pálffy András Gergely wrote:

...
Works here too. Great, thanks.

On Mon, Jun 28, 2010 at 1:25 PM, Nilesh Govindarajan wrote:

...
I have made a patch for /usr/share/hal/fdi/20-storage-methods.fdi to force async file transfer for vfat filesystems by commenting out flush and sync as valid options from the list. I checked the thing, now I'm getting the old high speed USB transfer. Do take a look at it, and comment.

Yes, but you should keep in mind that you'll spend extra time when you want to unmount your USB stick. So I prefer a "slow" transfer and a fast unmount, because usually I'm in hurry for taking off the USB drive and the unmounting visualizations are not very smart (only in KDE SC 4.4 it is really usable).

Actually I was recently wondering a bit about the unmounting part, especially with USB sticks. I do have udev rules, taken from the wiki, in place that handle automatic mounting. There's also a unmounting part, which afair removes created dirs, but I guess this is only called after the usb drive is removed. It did happen more than once to me that a file transfer seemed to be complete, but when I just removed the drive, the data was gone. Is there a way to provide automatic safe removal? Manual unmounting is a bit of a PITA, as you need to have a terminal ready, guess sdN and type a line, where the device guessing part is the most problematic. I tend to use /dev/sdN to make sure that I remove the device from all mount points. Thanks for any advice. -- Regards, Philipp -- "Wir stehen selbst enttäuscht und sehn betroffen / Den Vorhang zu und alle Fragen offen." Bertolt Brecht, Der gute Mensch von Sezuan

Denis A. Altoé Falqueto

10:09 p.m.

On Mon, Jun 28, 2010 at 5:52 PM, Philipp Überbacher wrote:

...

Actually I was recently wondering a bit about the unmounting part, especially with USB sticks. I do have udev rules, taken from the wiki, in place that handle automatic mounting. There's also a unmounting part, which afair removes created dirs, but I guess this is only called after the usb drive is removed. It did happen more than once to me that a file transfer seemed to be complete, but when I just removed the drive, the data was gone. Is there a way to provide automatic safe removal? Manual unmounting is a bit of a PITA, as you need to have a terminal ready, guess sdN and type a line, where the device guessing part is the most problematic. I tend to use /dev/sdN to make sure that I remove the device from all mount points. Thanks for any advice.

Well, automounting is really easy, but auto-unmounting (!?) is complex, because what triggers the event is the removal of the device, but after you take it, the S.O. can't do anything about it anymore. I use KDE, so I can mount and unmount easily. Gnome and XFCE also offer good services for that. But without those, I fear that you'll need to issue unmount manually. -- A: Because it obfuscates the reading. Q: Why is top posting so bad? ------------------------------------------- Denis A. Altoe Falqueto -------------------------------------------

Philipp Überbacher

10:27 p.m.

Excerpts from Denis A. Altoé Falqueto's message of 2010-06-29 00:09:11 +0200:

...

On Mon, Jun 28, 2010 at 5:52 PM, Philipp Überbacher wrote:

...
Actually I was recently wondering a bit about the unmounting part, especially with USB sticks. I do have udev rules, taken from the wiki, in place that handle automatic mounting. There's also a unmounting part, which afair removes created dirs, but I guess this is only called after the usb drive is removed. It did happen more than once to me that a file transfer seemed to be complete, but when I just removed the drive, the data was gone. Is there a way to provide automatic safe removal? Manual unmounting is a bit of a PITA, as you need to have a terminal ready, guess sdN and type a line, where the device guessing part is the most problematic. I tend to use /dev/sdN to make sure that I remove the device from all mount points. Thanks for any advice.

Well, automounting is really easy, but auto-unmounting (!?) is complex, because what triggers the event is the removal of the device, but after you take it, the S.O. can't do anything about it anymore. I use KDE, so I can mount and unmount easily. Gnome and XFCE also offer good services for that. But without those, I fear that you'll need to issue unmount manually.

That's what I thought. At this time I still use XFCEs thunar in case I want a graphical file manager, which uses HAL for mounting/unmounting, but I never managed to configure it correctly, so all I get is an error. But since HAL is gradually becoming unnecessary I don't want to bother with it. Mounting works well with udev alone. I was hoping that the device could, when inactive, be put into a state that prevents data loss. Not sure, kind of mount-on-demand maybe. I know far to little of what's going on to see a solution, I'm not even sure what causes the data loss when the writing is apparently finished. Indications for being finished is with my usb stick for example a differently glowing light and the indication in software of course. -- Regards, Philipp -- "Wir stehen selbst enttäuscht und sehn betroffen / Den Vorhang zu und alle Fragen offen." Bertolt Brecht, Der gute Mensch von Sezuan

Dave Reisner

11:18 p.m.

On Mon, Jun 28, 2010 at 07:09:11PM -0300, Denis A. Altoé Falqueto wrote:

...

On Mon, Jun 28, 2010 at 5:52 PM, Philipp Überbacher wrote:

...
Actually I was recently wondering a bit about the unmounting part, especially with USB sticks. I do have udev rules, taken from the wiki, in place that handle automatic mounting. There's also a unmounting part, which afair removes created dirs, but I guess this is only called after the usb drive is removed. It did happen more than once to me that a file transfer seemed to be complete, but when I just removed the drive, the data was gone. Is there a way to provide automatic safe removal? Manual unmounting is a bit of a PITA, as you need to have a terminal ready, guess sdN and type a line, where the device guessing part is the most problematic. I tend to use /dev/sdN to make sure that I remove the device from all mount points. Thanks for any advice.

Well, automounting is really easy, but auto-unmounting (!?) is complex, because what triggers the event is the removal of the device, but after you take it, the S.O. can't do anything about it anymore. I use KDE, so I can mount and unmount easily. Gnome and XFCE also offer good services for that. But without those, I fear that you'll need to issue unmount manually.

Not entirely true. You can assign the results of a blkid call to the sysfs node that correlates to the flash drive. A snip from my mounted flash drive.. ... E: SUBSYSTEM=block E: DEVNAME=sde1 E: ID_FS_LABEL=Flashy E: ID_FS_LABEL_ENC=Flashy E: ID_FS_UUID=6E1B5F1E742ED9F4 E: ID_FS_UUID_ENC=6E1B5F1E742ED9F4 E: ID_FS_TYPE=ntfs E: ID_FS_USAGE=filesystem ... The ID_FS_xxxx info is from blkid. On the REMOVE event, this information is still accessible, so I know where the drive is mounted because I always mount by label. When I remove the drive (without calling umount), the mount point is destroyed as well. Ensure that you're mounting the flash drive with 'sync' if you want to be able to remove it without calling umount. This will slow down transfers, but data is written synchronously to disk rather than to an intermediate buffer. As always, the Arch Wiki has examples of this on the Udev page. /dave

Javier Vasquez

29 Jun 29 Jun

12:11 a.m.

On 6/28/10, Dave Reisner wrote:

...

On Mon, Jun 28, 2010 at 07:09:11PM -0300, Denis A. Altoé Falqueto wrote:

...
On Mon, Jun 28, 2010 at 5:52 PM, Philipp Überbacher wrote:

...
Actually I was recently wondering a bit about the unmounting part, especially with USB sticks. I do have udev rules, taken from the wiki, in place that handle automatic mounting. There's also a unmounting part, which afair removes created dirs, but I guess this is only called after the usb drive is removed. It did happen more than once to me that a file transfer seemed to be complete, but when I just removed the drive, the data was gone. Is there a way to provide automatic safe removal? Manual unmounting is a bit of a PITA, as you need to have a terminal ready, guess sdN and type a line, where the device guessing part is the most problematic. I tend to use /dev/sdN to make sure that I remove the device from all mount points. Thanks for any advice.

Well, automounting is really easy, but auto-unmounting (!?) is complex, because what triggers the event is the removal of the device, but after you take it, the S.O. can't do anything about it anymore. I use KDE, so I can mount and unmount easily. Gnome and XFCE also offer good services for that. But without those, I fear that you'll need to issue unmount manually.

Not entirely true. You can assign the results of a blkid call to the sysfs node that correlates to the flash drive. A snip from my mounted flash drive..

... E: SUBSYSTEM=block E: DEVNAME=sde1 E: ID_FS_LABEL=Flashy E: ID_FS_LABEL_ENC=Flashy E: ID_FS_UUID=6E1B5F1E742ED9F4 E: ID_FS_UUID_ENC=6E1B5F1E742ED9F4 E: ID_FS_TYPE=ntfs E: ID_FS_USAGE=filesystem ...

The ID_FS_xxxx info is from blkid. On the REMOVE event, this information is still accessible, so I know where the drive is mounted because I always mount by label. When I remove the drive (without calling umount), the mount point is destroyed as well.

Ensure that you're mounting the flash drive with 'sync' if you want to be able to remove it without calling umount. This will slow down transfers, but data is written synchronously to disk rather than to an intermediate buffer.

As always, the Arch Wiki has examples of this on the Udev page.

/dave

To auto mount/umount usb kind of media without needing even X, you can use usbmount from debian: http://packages.debian.org/sid/usbmount Simple, light and pretty handy. However the umount part is done based upon removal, and the recommended/default way for mounting is with "sync" so that there's no data loss... -- Javier.

Ross

12:14 a.m.

On 29/06/10 08:52, Philipp Überbacher wrote:

...

Excerpts from Denis A. Altoé Falqueto's message of 2010-06-28 18:33:30 +0200:

...
On Mon, Jun 28, 2010 at 1:21 PM, Pálffy András Gergely wrote:

...
Works here too. Great, thanks.

On Mon, Jun 28, 2010 at 1:25 PM, Nilesh Govindarajanwrote:

...
I have made a patch for /usr/share/hal/fdi/20-storage-methods.fdi to force async file transfer for vfat filesystems by commenting out flush and sync as valid options from the list. I checked the thing, now I'm getting the old high speed USB transfer. Do take a look at it, and comment.

Yes, but you should keep in mind that you'll spend extra time when you want to unmount your USB stick. So I prefer a "slow" transfer and a fast unmount, because usually I'm in hurry for taking off the USB drive and the unmounting visualizations are not very smart (only in KDE SC 4.4 it is really usable).

Actually I was recently wondering a bit about the unmounting part, especially with USB sticks. I do have udev rules, taken from the wiki, in place that handle automatic mounting. There's also a unmounting part, which afair removes created dirs, but I guess this is only called after the usb drive is removed. It did happen more than once to me that a file transfer seemed to be complete, but when I just removed the drive, the data was gone. Is there a way to provide automatic safe removal? Manual unmounting is a bit of a PITA, as you need to have a terminal ready, guess sdN and type a line, where the device guessing part is the most problematic. I tend to use /dev/sdN to make sure that I remove the device from all mount points. Thanks for any advice.

I am no expert and am probably missing something here, but it should be simple to create a desktop icon and/or menu option to issue the sync command. That way you could have the speed of asynchronous mount and clicking the icon or menu option before removing the drive will write any buffered data to the device to prevent data loss if removing the device without umounting. As the sync command syncs all mounted drives you don't need to provide the /dev/sdN. Ross.

Nilesh Govindarajan

3:27 a.m.

On 06/29/2010 05:44 AM, Ross wrote:

...

On 29/06/10 08:52, Philipp Überbacher wrote:

...
Excerpts from Denis A. Altoé Falqueto's message of 2010-06-28 18:33:30 +0200:

...
On Mon, Jun 28, 2010 at 1:21 PM, Pálffy András Gergely wrote:

...
Works here too. Great, thanks.

On Mon, Jun 28, 2010 at 1:25 PM, Nilesh Govindarajanwrote:

...
I have made a patch for /usr/share/hal/fdi/20-storage-methods.fdi to force async file transfer for vfat filesystems by commenting out flush and sync as valid options from the list. I checked the thing, now I'm getting the old high speed USB transfer. Do take a look at it, and comment.

Yes, but you should keep in mind that you'll spend extra time when you want to unmount your USB stick. So I prefer a "slow" transfer and a fast unmount, because usually I'm in hurry for taking off the USB drive and the unmounting visualizations are not very smart (only in KDE SC 4.4 it is really usable).

Actually I was recently wondering a bit about the unmounting part, especially with USB sticks. I do have udev rules, taken from the wiki, in place that handle automatic mounting. There's also a unmounting part, which afair removes created dirs, but I guess this is only called after the usb drive is removed. It did happen more than once to me that a file transfer seemed to be complete, but when I just removed the drive, the data was gone. Is there a way to provide automatic safe removal? Manual unmounting is a bit of a PITA, as you need to have a terminal ready, guess sdN and type a line, where the device guessing part is the most problematic. I tend to use /dev/sdN to make sure that I remove the device from all mount points. Thanks for any advice.

I am no expert and am probably missing something here, but it should be simple to create a desktop icon and/or menu option to issue the sync command. That way you could have the speed of asynchronous mount and clicking the icon or menu option before removing the drive will write any buffered data to the device to prevent data loss if removing the device without umounting. As the sync command syncs all mounted drives you don't need to provide the /dev/sdN.

Ross.

Yeah exactly ! After copy your data to the drive, run the sync command or setup a keyboard shortcut for it :-) This will give you high speed transfer along with no data loss \m/ But if you're very forgetful to remove the drive without pressing the keyboard shortcut for sync, then you're in trouble and this patch is not for you. Alternatively, you could write a bash daemon as per this tutorial: http://j.mp/9DRWOF which will check for a usb stick's existence and if its mounted, sync every 15-30 seconds. -- Regards, Nilesh Govindarajan Facebook: http://www.facebook.com/nilesh.gr Twitter: http://twitter.com/nileshgr Website: http://www.itech7.com Cheap and Reliable VPS Hosting: http://j.mp/arHk5e

Philipp Überbacher

8:09 a.m.

Excerpts from Nilesh Govindarajan's message of 2010-06-29 05:27:42 +0200:

...

On 06/29/2010 05:44 AM, Ross wrote:

...
On 29/06/10 08:52, Philipp Überbacher wrote:

...
Excerpts from Denis A. Altoé Falqueto's message of 2010-06-28 18:33:30 +0200:

...
On Mon, Jun 28, 2010 at 1:21 PM, Pálffy András Gergely wrote:

...
Works here too. Great, thanks.

On Mon, Jun 28, 2010 at 1:25 PM, Nilesh Govindarajanwrote:

...
I have made a patch for /usr/share/hal/fdi/20-storage-methods.fdi to force async file transfer for vfat filesystems by commenting out flush and sync as valid options from the list. I checked the thing, now I'm getting the old high speed USB transfer. Do take a look at it, and comment.

Yes, but you should keep in mind that you'll spend extra time when you want to unmount your USB stick. So I prefer a "slow" transfer and a fast unmount, because usually I'm in hurry for taking off the USB drive and the unmounting visualizations are not very smart (only in KDE SC 4.4 it is really usable).

Actually I was recently wondering a bit about the unmounting part, especially with USB sticks. I do have udev rules, taken from the wiki, in place that handle automatic mounting. There's also a unmounting part, which afair removes created dirs, but I guess this is only called after the usb drive is removed. It did happen more than once to me that a file transfer seemed to be complete, but when I just removed the drive, the data was gone. Is there a way to provide automatic safe removal? Manual unmounting is a bit of a PITA, as you need to have a terminal ready, guess sdN and type a line, where the device guessing part is the most problematic. I tend to use /dev/sdN to make sure that I remove the device from all mount points. Thanks for any advice.

I am no expert and am probably missing something here, but it should be simple to create a desktop icon and/or menu option to issue the sync command. That way you could have the speed of asynchronous mount and clicking the icon or menu option before removing the drive will write any buffered data to the device to prevent data loss if removing the device without umounting. As the sync command syncs all mounted drives you don't need to provide the /dev/sdN.

Ross.

Yeah exactly ! After copy your data to the drive, run the sync command or setup a keyboard shortcut for it :-) This will give you high speed transfer along with no data loss \m/

But if you're very forgetful to remove the drive without pressing the keyboard shortcut for sync, then you're in trouble and this patch is not for you.

Alternatively, you could write a bash daemon as per this tutorial: http://j.mp/9DRWOF

which will check for a usb stick's existence and if its mounted, sync every 15-30 seconds.

Heh, I didn't even know about the sync command and not about the difference between sync/async either. Guess I'll have to check my mount options and udev rules. The bash daemon tutorial thing looks handy and simple enough too, so thanks! -- Regards, Philipp -- "Wir stehen selbst enttäuscht und sehn betroffen / Den Vorhang zu und alle Fragen offen." Bertolt Brecht, Der gute Mensch von Sezuan

5058

Age (days ago)

5059

Last active (days ago)

List overview

Download

10 comments

7 participants

participants (7)

Dave Reisner
Denis A. Altoé Falqueto
Javier Vasquez
Nilesh Govindarajan
Philipp Überbacher
Pálffy András Gergely
Ross