[arch-general] Mounted ext4 filesystem without journal
Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist. The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs. Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition. Any pointers on how to proceed with this would be much appreciated.
I haven't had any replies to the email below so far. Not sure if that's because that email didn't make it through to the mailing list, even though it does show up here <https://lists.archlinux.org/pipermail/arch-general/2021-February/048597.html>, or if it's because the problem I'm asking help for is unique and the members of the list have nothing to suggest. Thought I'd resend the original email just in case. Any pointers would be appreciated. ---------- Forwarded message --------- From: Alexander Kapshuk <alexander.kapshuk@gmail.com> Date: Thu, Feb 4, 2021 at 5:40 PM Subject: Mounted ext4 filesystem without journal To: General Discussion about Arch Linux <arch-general@archlinux.org> Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist. The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs. Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition. Any pointers on how to proceed with this would be much appreciated.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256 Hi, On Sun, 7 Feb 2021, Alexander Kapshuk via arch-general wrote:
I haven't had any replies to the email below so far. Not sure if that's because that email didn't make it through to the mailing list, even though it does show up here <https://lists.archlinux.org/pipermail/arch-general/2021-February/048597.html>, or if it's because the problem I'm asking help for is unique and the members of the list have nothing to suggest.
Thought I'd resend the original email just in case. Any pointers would be appreciated.
---------- Forwarded message --------- From: Alexander Kapshuk <alexander.kapshuk@gmail.com> Date: Thu, Feb 4, 2021 at 5:40 PM Subject: Mounted ext4 filesystem without journal To: General Discussion about Arch Linux <arch-general@archlinux.org>
Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist.
looks, like fsck emptied your disk (or at least the directory structure is gone) - most probably, because it was already severely damaged before. Why do you suspect, your kernel has anything to do with it? I would rather suspect a hardware error of some kind (/dev/sda is a plain ssd or hdd - not some raid or cryptdevice, I assume).
The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs.
Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition.
Lost inodes end up in lost+found/ when you run fsck (at least, that's, what has happened to me before). If nothing is left outside of lost+found/, I suggest: 1. Use your last backup. - You do have a backup, right? If not: 2. copy lost+found/ somewhere safe, do a fresh install, and have a look at your saved inodes for valuable content (not sure, if there are some tools to help you with skimming through lots of unnamed inodes - usually at least *one* of my backups was good enough for me in the past).
Any pointers on how to proceed with this would be much appreciated.
Use backups :-) regards, Erich -----BEGIN PGP SIGNATURE----- iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAmAgMFUACgkQCu7JB1Xa e1pTuhAAvMXySI3caWjB+/UQZ81+yhvajVuO2JuLwmsncJTRjL63dEwZzKObgtdu yL6HacceXVdzPyqDxOtuCYL6e62HIV+hf8MB6Jza00n0HCJp0CEVMRwnMjiMQPkk Io9gQj7S6cCR5IPEIuC0pn/jQxGQOrbP/Jom6YNtc0qQ6xzaaRJ4w7RylqnbBFTJ 7StS62J645U94905a8eY1Nuwr5KawQFQEi1Ar/RRL8M/3dd5iCMXR7MRynawLj4l cNAv9kSgocZ+RlL61NywU9zyCZbNMnRurHfgeaCo/DudMjHPEqWYr3R0BYo4wWDJ cjDjby+oHdKw0eQqhCkHlPT+ocnLtSvoJmCmyimbeNDNRcOrvyd8Ni+2GYpgYw9d 7t/lO1WI63PlvngxDE17L1pzirafgqJmHo+QnYrwIQGuANjqaDZBoDIzQqscTb12 84Yh++kZbdxIOQ+ZhM1LqRd58UwZkiTxWm20yyOrBX720gYRMqJ6HF6AU5SzKFt0 IAXIIZXFNrRS5xuHl2c57jWwutnFQaHXoM8j/0BcnCTgJEooUs0+MbqjbEYG+a8Q wrtx23uvdj4Qy/70q0G5dQoZVANwTDKUWVqyGFlWFCp5yzfIrZkQbdWVjjDBW0jT Mwg+fVP+/DtghulxQRB9QheV+wpJSVAmxlcoqKCGr2xe08697qM= =TiKP -----END PGP SIGNATURE-----
On Sun, Feb 7, 2021 at 8:24 PM Erich Eckner via arch-general <arch-general@lists.archlinux.org> wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA256
Hi,
On Sun, 7 Feb 2021, Alexander Kapshuk via arch-general wrote:
I haven't had any replies to the email below so far. Not sure if that's because that email didn't make it through to the mailing list, even though it does show up here <https://lists.archlinux.org/pipermail/arch-general/2021-February/048597.html>, or if it's because the problem I'm asking help for is unique and the members of the list have nothing to suggest.
Thought I'd resend the original email just in case. Any pointers would be appreciated.
---------- Forwarded message --------- From: Alexander Kapshuk <alexander.kapshuk@gmail.com> Date: Thu, Feb 4, 2021 at 5:40 PM Subject: Mounted ext4 filesystem without journal To: General Discussion about Arch Linux <arch-general@archlinux.org>
Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist.
looks, like fsck emptied your disk (or at least the directory structure is gone) - most probably, because it was already severely damaged before. Why do you suspect, your kernel has anything to do with it? I would rather suspect a hardware error of some kind (/dev/sda is a plain ssd or hdd - not some raid or cryptdevice, I assume).
I had fsck complain about /dev/sda2 on two machines on booting the same version of linux-next. Fsck was able to fix the errors reported on one machine, but not on the other. /dev/sda is a hdd.
The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs.
Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition.
Lost inodes end up in lost+found/ when you run fsck (at least, that's, what has happened to me before). If nothing is left outside of lost+found/, I suggest:
1. Use your last backup. - You do have a backup, right? If not:
2. copy lost+found/ somewhere safe, do a fresh install, and have a look at your saved inodes for valuable content (not sure, if there are some tools to help you with skimming through lots of unnamed inodes - usually at least *one* of my backups was good enough for me in the past).
Any pointers on how to proceed with this would be much appreciated.
Use backups :-)
regards, Erich
Thanks for your feedback.
-----BEGIN PGP SIGNATURE-----
iQIzBAEBCAAdFiEE3p92iMrPBP64GmxZCu7JB1Xae1oFAmAgMFUACgkQCu7JB1Xa e1pTuhAAvMXySI3caWjB+/UQZ81+yhvajVuO2JuLwmsncJTRjL63dEwZzKObgtdu yL6HacceXVdzPyqDxOtuCYL6e62HIV+hf8MB6Jza00n0HCJp0CEVMRwnMjiMQPkk Io9gQj7S6cCR5IPEIuC0pn/jQxGQOrbP/Jom6YNtc0qQ6xzaaRJ4w7RylqnbBFTJ 7StS62J645U94905a8eY1Nuwr5KawQFQEi1Ar/RRL8M/3dd5iCMXR7MRynawLj4l cNAv9kSgocZ+RlL61NywU9zyCZbNMnRurHfgeaCo/DudMjHPEqWYr3R0BYo4wWDJ cjDjby+oHdKw0eQqhCkHlPT+ocnLtSvoJmCmyimbeNDNRcOrvyd8Ni+2GYpgYw9d 7t/lO1WI63PlvngxDE17L1pzirafgqJmHo+QnYrwIQGuANjqaDZBoDIzQqscTb12 84Yh++kZbdxIOQ+ZhM1LqRd58UwZkiTxWm20yyOrBX720gYRMqJ6HF6AU5SzKFt0 IAXIIZXFNrRS5xuHl2c57jWwutnFQaHXoM8j/0BcnCTgJEooUs0+MbqjbEYG+a8Q wrtx23uvdj4Qy/70q0G5dQoZVANwTDKUWVqyGFlWFCp5yzfIrZkQbdWVjjDBW0jT Mwg+fVP+/DtghulxQRB9QheV+wpJSVAmxlcoqKCGr2xe08697qM= =TiKP -----END PGP SIGNATURE-----
Alexander Kapshuk via arch-general <arch-general@lists.archlinux.org> wrote:
I haven't had any replies to the email below so far. Not sure if that's because that email didn't make it through to the mailing list, even though it does show up here <https://lists.archlinux.org/pipermail/arch-general/2021-February/048597.html>, or if it's because the problem I'm asking help for is unique and the members of the list have nothing to suggest.
Thought I'd resend the original email just in case. Any pointers would be appreciated.
---------- Forwarded message --------- From: Alexander Kapshuk <alexander.kapshuk@gmail.com> Date: Thu, Feb 4, 2021 at 5:40 PM Subject: Mounted ext4 filesystem without journal To: General Discussion about Arch Linux <arch-general@archlinux.org>
Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist.
The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs.
Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition.
Any pointers on how to proceed with this would be much appreciated.
Hello, I recieved your original message throught the list. Other then suggesting to check for bad blocks and the S.M.A.R.T data of the device, and restore from backup, I have no ideas. -- u34
On Sun, Feb 7, 2021 at 8:26 PM u34--- via arch-general <arch-general@lists.archlinux.org> wrote:
Alexander Kapshuk via arch-general <arch-general@lists.archlinux.org> wrote:
I haven't had any replies to the email below so far. Not sure if that's because that email didn't make it through to the mailing list, even though it does show up here <https://lists.archlinux.org/pipermail/arch-general/2021-February/048597.html>, or if it's because the problem I'm asking help for is unique and the members of the list have nothing to suggest.
Thought I'd resend the original email just in case. Any pointers would be appreciated.
---------- Forwarded message --------- From: Alexander Kapshuk <alexander.kapshuk@gmail.com> Date: Thu, Feb 4, 2021 at 5:40 PM Subject: Mounted ext4 filesystem without journal To: General Discussion about Arch Linux <arch-general@archlinux.org>
Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist.
The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs.
Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition.
Any pointers on how to proceed with this would be much appreciated.
Hello,
I recieved your original message throught the list. Other then suggesting to check for bad blocks and the S.M.A.R.T data of the device, and restore from backup, I have no ideas.
-- u34
Thanks for your feedback. I'll give that a try.
There is a tool called testdisk (https://www.cgsecurity.org) that will scan all blocks of your disk and try to interprete whatever leftovers of the filesystem it finds. It might be able to recover some parts but you will need a second disk as recovery target. Good luck. Am 07.02.21 um 18:39 schrieb Alexander Kapshuk via arch-general:
I haven't had any replies to the email below so far. Not sure if that's because that email didn't make it through to the mailing list, even though it does show up here <https://lists.archlinux.org/pipermail/arch-general/2021-February/048597.html>, or if it's because the problem I'm asking help for is unique and the members of the list have nothing to suggest.
Thought I'd resend the original email just in case. Any pointers would be appreciated.
---------- Forwarded message --------- From: Alexander Kapshuk <alexander.kapshuk@gmail.com> Date: Thu, Feb 4, 2021 at 5:40 PM Subject: Mounted ext4 filesystem without journal To: General Discussion about Arch Linux <arch-general@archlinux.org>
Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist.
The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs.
Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition.
Any pointers on how to proceed with this would be much appreciated.
On Sun, Feb 7, 2021 at 8:45 PM Uwe Sauter via arch-general <arch-general@lists.archlinux.org> wrote:
There is a tool called testdisk (https://www.cgsecurity.org) that will scan all blocks of your disk and try to interprete whatever leftovers of the filesystem it finds. It might be able to recover some parts but you will need a second disk as recovery target.
Good luck.
Thanks for your feedback. I'll give it a go.
Am 07.02.21 um 18:39 schrieb Alexander Kapshuk via arch-general:
I haven't had any replies to the email below so far. Not sure if that's because that email didn't make it through to the mailing list, even though it does show up here <https://lists.archlinux.org/pipermail/arch-general/2021-February/048597.html>, or if it's because the problem I'm asking help for is unique and the members of the list have nothing to suggest.
Thought I'd resend the original email just in case. Any pointers would be appreciated.
---------- Forwarded message --------- From: Alexander Kapshuk <alexander.kapshuk@gmail.com> Date: Thu, Feb 4, 2021 at 5:40 PM Subject: Mounted ext4 filesystem without journal To: General Discussion about Arch Linux <arch-general@archlinux.org>
Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist.
The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs.
Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition.
Any pointers on how to proceed with this would be much appreciated.
‐‐‐‐‐‐‐ Original Message ‐‐‐‐‐‐‐ On Thursday, 4 February 2021 г., 18:40, Alexander Kapshuk via arch-general <arch-general@lists.archlinux.org> wrote:
Most probably due to a buggy custom kernel, fsck on /dev/sda2, i.e the root partition, failed with a rootfs prompt and an invitation to run fsck on /dev/sda2 presented. On having done that and rebooted, when performing fsck on /dev/sda2, the system reported that: /dev/sda2 had not been cleanly unmounted 0.4% non-contiguous blocks mounting /dev/sda2 on real root EXT4-fs (sda2) mounted filesystem without journal Root device mounted successfully, but /sbin/init does not exist.
The wiki article on fsck troubleshooting, https://wiki.archlinux.org/index.php/Fsck#Troubleshooting, suggests using tune2fs to write a new journal to /dev/sda2. But it doesn't seem to be available when in rootfs.
Booting into the Arch Linux installation CD and mounting /dev/sda2 over /mnt reveals lost+found as the sole contents of the directory. As an experiment, mounting /dev/sda1 over /mnt confirms the presence of all the files there are on my boot partition.
Any pointers on how to proceed with this would be much appreciated.
Judging by absense of init and only 'lost+found' folder it seems that your root partition is totally damaged. It is unclear from your report whether it was damaged before fsck or during this operation. Anyway, you can try any data restore utility or ext4 tools to try to repair the data. As others already suggested, you should also test your disk with smarttools (probably RAM also). P.S. Your case is somewhat strange. Basically you have some unknown issue with root fs and agreed to fsck it. After reboot your root fs looks fine, but all data is gone - this does not look like typical fs corruption where fs metadata is damaged and partition cannot be mounted (and it does contain some 'artefacts' or garbage data).
participants (5)
-
Alexander Kapshuk
-
Erich Eckner
-
Maksim Fomin
-
u34＠net9.ga
-
Uwe Sauter