Hi everyone, I am currently having an issue with the new NTFS module. I am able to mount, browse the partition and create files; however, when writing content into a file, that does not happen. It seems the content is written only at partition unmount. I tried mounting my partition with both *mount* and *udisksctl* with the *ntfs3* module and it didn't work. The old *ntfs-3g* module works perfectly. Any clues on what it is happening? I can provide any logs you wish. Giovanni Santini
Am Mittwoch, 15. November 2023, 12:59:27 CET schrieb Giovanni Santini:
I am able to mount, browse the partition and create files; however, when writing content into a file, that does not happen.
It seems the content is written only at partition unmount.
Since both are completely different implementations the syncing behavior might differ. I haven't paid much attention to the syncing behavior of the new kernel module but it might simply be more lazy than the FUSE implementation. Does it help to invoke `sync` manually? If yes, this may also be a workaround in case you really want it to write the data without delay.
Hi Marius, On 2023-11-15 14:16, Marius Kittler wrote:
Since both are completely different implementations the syncing behavior might differ. I haven't paid much attention to the syncing behavior of the new kernel module but it might simply be more lazy than the FUSE implementation. Does it help to invoke `sync` manually? If yes, this may also be a workaround in case you really want it to write the data without delay.
I did try, yes. No fortune with manual sync and udisks or mount. Here is a shell session, I think a pastebin is not needed given that it is short: --- (23:16) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (23:16) giovanni @ ~ $ touch /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ ls -al /run/media/giovanni/Data/ntfs_error.txt -rw-r--r-- 1 giovanni users 0 Nov 16 23:17 /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ echo "Using ntfs3" > /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ cat /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ udisksctl unmount -b /dev/nvme0n1p5 Unmounted /dev/nvme0n1p5. (23:17) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (23:17) giovanni @ ~ $ cat /run/media/giovanni/Data/ntfs_error.txt Using ntfs3 (23:17) giovanni @ ~ $ udisksctl unmount -b /dev/nvme0n1p5 Unmounted /dev/nvme0n1p5. (23:18) giovanni @ ~ $ sudo mount -t ntfs3 -o uid=$UID,gid=$GROUPS /dev/nvme0n1p5 /mnt/ (23:18) giovanni @ ~ $ cat /mnt/ntfs_error.txt Using ntfs3 (23:18) giovanni @ ~ $ echo "Using ntfs3 and mount" > /mnt/ntfs_error.txt (23:19) giovanni @ ~ $ cat /mnt/ntfs_error.txt (23:19) giovanni @ ~ $ sync (23:19) giovanni @ ~ $ cat /mnt/ntfs_error.txt (23:19) giovanni @ ~ $ sync (23:19) giovanni @ ~ $ cat /mnt/ntfs_error.txt (23:19) giovanni @ ~ $ sudo umount /mnt (23:19) giovanni @ ~ $ sudo mount -t ntfs3 -o uid=$UID,gid=$GROUPS /dev/nvme0n1p5 /mnt/ (23:19) giovanni @ ~ $ cat /mnt/ntfs_error.txt Using ntfs3 and mount (23:19) giovanni @ ~ $ --- You can see that when I try to overwrite the file content it just "appears" empty. Unmounting and remounting the partition makes the content appear. Bests, -- Giovanni Santini
Le ven. 17 nov. 2023 à 07:41, Giovanni Santini <giovannisantini93@yahoo.it> a écrit :
Hi Marius, On 2023-11-15 14:16, Marius Kittler wrote:
Since both are completely different implementations the syncing behavior might differ. I haven't paid much attention to the syncing behavior of the new kernel module but it might simply be more lazy than the FUSE implementation. Does it help to invoke `sync` manually? If yes, this may also be a workaround in case you really want it to write the data without delay.
I did try, yes. No fortune with manual sync and udisks or mount.
Here is a shell session, I think a pastebin is not needed given that it is short:
---
(23:16) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (23:16) giovanni @ ~ $ touch /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ ls -al /run/media/giovanni/Data/ntfs_error.txt -rw-r--r-- 1 giovanni users 0 Nov 16 23:17 /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ echo "Using ntfs3" > /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ cat /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ udisksctl unmount -b /dev/nvme0n1p5 Unmounted /dev/nvme0n1p5. (23:17) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (23:17) giovanni @ ~ $ cat /run/media/giovanni/Data/ntfs_error.txt Using ntfs3 (23:17) giovanni @ ~ $ udisksctl unmount -b /dev/nvme0n1p5 Unmounted /dev/nvme0n1p5. (23:18) giovanni @ ~ $ sudo mount -t ntfs3 -o uid=$UID,gid=$GROUPS /dev/nvme0n1p5 /mnt/ (23:18) giovanni @ ~ $ cat /mnt/ntfs_error.txt Using ntfs3 (23:18) giovanni @ ~ $ echo "Using ntfs3 and mount" > /mnt/ntfs_error.txt (23:19) giovanni @ ~ $ cat /mnt/ntfs_error.txt (23:19) giovanni @ ~ $ sync (23:19) giovanni @ ~ $ cat /mnt/ntfs_error.txt (23:19) giovanni @ ~ $ sync (23:19) giovanni @ ~ $ cat /mnt/ntfs_error.txt (23:19) giovanni @ ~ $ sudo umount /mnt (23:19) giovanni @ ~ $ sudo mount -t ntfs3 -o uid=$UID,gid=$GROUPS /dev/nvme0n1p5 /mnt/ (23:19) giovanni @ ~ $ cat /mnt/ntfs_error.txt Using ntfs3 and mount (23:19) giovanni @ ~ $
---
You can see that when I try to overwrite the file content it just "appears" empty.
Unmounting and remounting the partition makes the content appear.
Bests,
-- Giovanni Santini
on the `/proc/sys/vm/` side may be ? does the `sync` followed by a `sysctl -q vm.drop_caches=3` change things ? regards, lacsaP.
Hi IacsaP and Lukasz, On 2023-11-17 08:40, lacsaP Patatetom wrote:
on the `/proc/sys/vm/` side may be ? does the `sync` followed by a `sysctl -q vm.drop_caches=3` change things ?
regards, lacsaP.
tried but to no avail. Here is the session output: --- (13:44) giovanni @ ~ $ sysctl vm.drop_caches (13:44) giovanni @ ~ $ sudo sysctl vm.drop_caches=3 [sudo] password for giovanni: vm.drop_caches = 3 (13:44) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (13:44) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt Using ntfs3 (13:45) giovanni @ ~ $ echo "Using udisks and ntfs3" > /run/media/giovanni/Data/mount_test.txt (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt (13:45) giovanni @ ~ $ sync (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt (13:45) giovanni @ ~ $ udisksctl unmount -b /dev/nvme0n1p5 Unmounted /dev/nvme0n1p5. (13:45) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt Using udisks and ntfs3 (13:45) giovanni @ ~ $ --- As Lukasz mentioned, it probably is a bug. I will open one on the bug tracker and if needed towards the kernel itself. :) Will keep you posted. Bests, -- Giovanni Santini
Hi IacsaP and Lukasz, On 2023-11-17 08:40, lacsaP Patatetom wrote:
on the `/proc/sys/vm/` side may be ? does the `sync` followed by a `sysctl -q vm.drop_caches=3` change things ?
regards, lacsaP.
tried but to no avail.
Here is the session output:
---
(13:44) giovanni @ ~ $ sysctl vm.drop_caches (13:44) giovanni @ ~ $ sudo sysctl vm.drop_caches=3 [sudo] password for giovanni: vm.drop_caches = 3 (13:44) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (13:44) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt Using ntfs3 (13:45) giovanni @ ~ $ echo "Using udisks and ntfs3" > /run/media/giovanni/Data/mount_test.txt (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt (13:45) giovanni @ ~ $ sync (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt (13:45) giovanni @ ~ $ udisksctl unmount -b /dev/nvme0n1p5 Unmounted /dev/nvme0n1p5. (13:45) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt Using udisks and ntfs3 (13:45) giovanni @ ~ $
---
As Lukasz mentioned, it probably is a bug.
I will open one on the bug tracker and if needed towards the kernel itself. :)
Will keep you posted.
Bests,
-- Giovanni Santini
(13:45) giovanni @ ~ $ echo "Using udisks and ntfs3" > /run/media/giovanni/Data/mount_test.txt (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt
Le ven. 17 nov. 2023 à 13:49, Giovanni Santini <giovannisantini93@yahoo.it> a écrit : try calling `sysctl` after `sync` here : (13:45) giovanni @ ~ $ sync; sudo sysctl vm.drop_caches=3 (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt
Hi IacsaP, On 2023-11-17 14:20, lacsaP Patatetom wrote:
(13:45) giovanni @ ~ $ echo "Using udisks and ntfs3" > /run/media/giovanni/Data/mount_test.txt (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt try calling `sysctl` after `sync` here : (13:45) giovanni @ ~ $ sync; sudo sysctl vm.drop_caches=3 (13:45) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt
Here is the result: --- (17:17) giovanni @ ~ $ udisksctl mount -b /dev/nvme0n1p5 -t ntfs3 Mounted /dev/nvme0n1p5 at /run/media/giovanni/Data (17:17) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt Using udisks and ntfs3 (17:17) giovanni @ ~ $ echo "Using udisks and ntfs3" > /run/media/giovanni/Data/mount_test.txt (17:18) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt (17:18) giovanni @ ~ $ sync; sudo sysctl vm.drop_caches=3 vm.drop_caches = 3 (17:18) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt Using udisks and ntfs3 (17:18) giovanni @ ~ $ echo "Using udisks and ntfs3 again" > /run/media/giovanni/Data/mount_test.txt (17:18) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt (17:18) giovanni @ ~ $ sync (17:18) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt (17:18) giovanni @ ~ $ sync; sudo sysctl vm.drop_caches=3 vm.drop_caches = 3 (17:19) giovanni @ ~ $ cat /run/media/giovanni/Data/mount_test.txt Using udisks and ntfs3 again (17:19) giovanni @ ~ $ --- Having to always run the sysctl command in order to force the flush seems very bad to me... So, again, sync alone does not work. sysctl then sync does not. sync and then sysctl does... But the sysctl command needs to be executed every time. -- Giovanni Santini
On 11/16/23 23:23, Giovanni Santini wrote:
Hi Marius, On 2023-11-15 14:16, Marius Kittler wrote:
Since both are completely different implementations the syncing behavior might differ. I haven't paid much attention to the syncing behavior of the new kernel module but it might simply be more lazy than the FUSE implementation. Does it help to invoke `sync` manually? If yes, this may also be a workaround in case you really want it to write the data without delay.
I did try, yes. No fortune with manual sync and udisks or mount.
Here is a shell session, I think a pastebin is not needed given that it is short:
---
(23:16) giovanni @ ~ $ touch /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ ls -al /run/media/giovanni/Data/ntfs_error.txt -rw-r--r-- 1 giovanni users 0 Nov 16 23:17 /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ echo "Using ntfs3" > /run/media/giovanni/Data/ntfs_error.txt (23:17) giovanni @ ~ $ cat /run/media/giovanni/Data/ntfs_error.txt
Looks like a bug. If something is written to file you should be able to read it afterwards no matter if content is still in i/o buffer or already written to disk. sync won't help here. Regards, Łukasz
Hi everyone, On 2023-11-17 11:13, Łukasz Michalski wrote:
Looks like a bug. If something is written to file you should be able to read it afterwards no matter if content is still in i/o buffer or already written to disk.
This is definitely correct. I just verified that `linux-lts` does not have the issue at all. Let's hope the problem gets solved soon. :) -- Giovanni Santini
On 11/18/23 07:18, Giovanni Santini wrote: ...
Let's hope the problem gets solved soon. :)
Hi You may have mentioned this already, but is there anything interesting in the kernel logs? I see no relevant looking ntfs3 commits in the kernel after 6.6. There were some in 6.6 [1]. Since linux-lts (6.1) is behaving differently for you than 6.6.1 you may want to report this upstream [2]. It could be that the above commit is related to your problem - you could try building a 6.6.1 kernel with it reverted and see if things work. If not, maybe do git bisect if you're comfortable doing that. You may also want to be aware of: https://lore.kernel.org/ntfs3/ gene [1] commit f69d00d12fcee9ce4b4f24bf609ad7398d10a576 Merge: 7cf4bea77ab6 e4494770a5ca Author: Linus Torvalds <torvalds@linux-foundation.org> Date: Thu Oct 19 09:10:18 2023 -0700 Merge tag 'ntfs3_for_6.6' of https://github.com/Paragon-Software-Group/linux-ntfs3 [2] https://www.kernel.org/doc/html/v4.19/admin-guide/reporting-bugs.html. N.B. bugzilla.kernel.org is now monitored and bugs are pulled out and emailed to maintainers and tracked.
Hi everyone again, I started opening an issue on the Arch bug tracker [1] and on the kernel one [2]. My main question is: does anyone else have the same issue? This is because the person helping me at the Arch bug tracker is unable to replicate it... I am wondering what sort of weird combination makes this to happen. Bests, Giovanni [1]: https://bugs.archlinux.org/task/80283 [2]: https://bugzilla.kernel.org/show_bug.cgi?id=218180 -- Giovanni Santini
participants (5)
-
Genes Lists
-
Giovanni Santini
-
lacsaP Patatetom
-
Marius Kittler
-
Łukasz Michalski