[arch-general] Hibernation Failure
I'm having trouble getting hibernation to work on my new Dell Inspiron 7590. It seems that the image is stored correctly (onto LUKS+LVM thin-lv), but resume fails, with the following left in the kernel logs: --- PM: Image signature found, resuming PM: resume from hibernation Freezing user space processes ... (elapsed 0.001 seconds) done. OOM killer disabled. PM: Marking nosave pages: [mem 0x00000000-0x00000fff] PM: Marking nosave pages: [mem 0x0009e000-0x0009efff] PM: Marking nosave pages: [mem 0x000a0000-0x000fffff] PM: Marking nosave pages: [mem 0x40000000-0x403fffff] PM: Marking nosave pages: [mem 0x66797000-0x66798fff] PM: Marking nosave pages: [mem 0x761f7000-0x791fefff] PM: Marking nosave pages: [mem 0x79200000-0xffffffff] PM: Basic memory bitmaps created PM: Using 3 thread(s) for decompression PM: Loading and decompressing image data (1343374 pages)... PM: Image loading progress: 0% PM: Image loading progress: 10% PM: Image loading progress: 20% PM: Image loading progress: 30% PM: Image loading progress: 40% PM: Image loading progress: 50% PM: Image loading progress: 60% PM: Image loading progress: 70% PM: Image loading progress: 80% PM: Image loading progress: 90% PM: Invalid LZO compressed length PM: Read 5373496 kbytes in 6.35 seconds (846.21 MB/s) PM: Error -1 resuming PM: Failed to load hibernation image, recovering. PM: Basic memory bitmaps freed OOM killer enabled. Restarting tasks ... done. PM: resume from hibernation failed (-1) --- And then the system boots fresh. An internet search revealed absolutely nothing about the "Invalid LZO compressed length", and I've no idea why the image would be corrupt. I'd appreciate any ideas. Paul
https://wiki.archlinux.org/index.php/Dm-crypt/Swap_encryption#With_suspend-t... On Sat, Jan 11, 2020 at 4:40 PM Paul Dann via arch-general <arch-general@archlinux.org> wrote:
I'm having trouble getting hibernation to work on my new Dell Inspiron 7590. It seems that the image is stored correctly (onto LUKS+LVM thin-lv), but resume fails, with the following left in the kernel logs:
--- PM: Image signature found, resuming PM: resume from hibernation Freezing user space processes ... (elapsed 0.001 seconds) done. OOM killer disabled. PM: Marking nosave pages: [mem 0x00000000-0x00000fff] PM: Marking nosave pages: [mem 0x0009e000-0x0009efff] PM: Marking nosave pages: [mem 0x000a0000-0x000fffff] PM: Marking nosave pages: [mem 0x40000000-0x403fffff] PM: Marking nosave pages: [mem 0x66797000-0x66798fff] PM: Marking nosave pages: [mem 0x761f7000-0x791fefff] PM: Marking nosave pages: [mem 0x79200000-0xffffffff] PM: Basic memory bitmaps created PM: Using 3 thread(s) for decompression PM: Loading and decompressing image data (1343374 pages)... PM: Image loading progress: 0% PM: Image loading progress: 10% PM: Image loading progress: 20% PM: Image loading progress: 30% PM: Image loading progress: 40% PM: Image loading progress: 50% PM: Image loading progress: 60% PM: Image loading progress: 70% PM: Image loading progress: 80% PM: Image loading progress: 90% PM: Invalid LZO compressed length PM: Read 5373496 kbytes in 6.35 seconds (846.21 MB/s) PM: Error -1 resuming PM: Failed to load hibernation image, recovering. PM: Basic memory bitmaps freed OOM killer enabled. Restarting tasks ... done. PM: resume from hibernation failed (-1) ---
And then the system boots fresh. An internet search revealed absolutely nothing about the "Invalid LZO compressed length", and I've no idea why the image would be corrupt. I'd appreciate any ideas.
Paul
On Sun, 12 Jan 2020, 10:08 Justin Capella via arch-general, < arch-general@archlinux.org> wrote:
https://wiki.archlinux.org/index.php/Dm-crypt/Swap_encryption#With_suspend-t...
Nope; if that was the problem then the kernel would not see the resume data at all, because the swap block device would not yet exist. As you can see from the log I posted, the kernel recognises the data and attempts to decompress it, so the issue is not so straightforward. Paul
Am 12.01.20 um 01:39 schrieb Paul Dann via arch-general:
I'm having trouble getting hibernation to work on my new Dell Inspiron 7590. It seems that the image is stored correctly (onto LUKS+LVM thin-lv),
I've noticed something similar on a similar setup. It looks like the kernel is taking some shortcuts when accessing the (swap-) space during hibernation, which are not compatible with dm-crypt. I'm using an AEAD cipher and the integrity data is wrong after resume. BR
Ah interesting. Do those memory regions match what is reported in proc/iomem prior to hibernating-- odd question but I recall someone on the irc encountering a difference. I suggest hopping on the IRC if you're able, easier and quicker, probably not arch specific but feel free to start there or maybe the ##kernel channel On Sun, Jan 12, 2020, 3:19 AM Markus Schaaf via arch-general < arch-general@archlinux.org> wrote:
I'm having trouble getting hibernation to work on my new Dell Inspiron 7590. It seems that the image is stored correctly (onto LUKS+LVM
Am 12.01.20 um 01:39 schrieb Paul Dann via arch-general: thin-lv),
I've noticed something similar on a similar setup. It looks like the kernel is taking some shortcuts when accessing the (swap-) space during hibernation, which are not compatible with dm-crypt. I'm using an AEAD cipher and the integrity data is wrong after resume.
BR
On Sun, 12 Jan 2020 at 12:03, Justin Capella via arch-general < arch-general@archlinux.org> wrote:
Ah interesting. Do those memory regions match what is reported in proc/iomem prior to hibernating-- odd question but I recall someone on the irc encountering a difference.
Mmm that *is* interesting. The reserved sections are mostly a match, but not 100%: -- 00000000-00000fff 0009e000-0009efff 000a0000-000fffff 40000000-403fffff 66798000-66798fff 761f7000-7796efff 78361000-7901cfff 79200000-7f7fffff fec00000-fec00fff ff000000-ffffffff ---
I suggest hopping on the IRC if you're able, easier and quicker, probably not arch specific but feel free to start there or maybe the ##kernel channel
I'll give that a go; thanks. Paul
Am 12.01.20 um 12:19 schrieb Markus Schaaf via arch-general:
Am 12.01.20 um 01:39 schrieb Paul Dann via arch-general:
I'm having trouble getting hibernation to work on my new Dell Inspiron 7590. It seems that the image is stored correctly (onto LUKS+LVM thin-lv),
I've noticed something similar on a similar setup. It looks like the kernel is taking some shortcuts when accessing the (swap-) space during hibernation, which are not compatible with dm-crypt. I'm using an AEAD cipher and the integrity data is wrong after resume.
I know it is terribly late, but for those curious: While investigating this I managed to make my laptop unbootable, because dm-crypt decided that every single sector on my encrypted partition had a bad AEAD tag and wouldn't let me read a single byte. That was exactly what had been happening to my swap-partition before, when I tried to resume from hibernation. But this time it had eaten my root-partition too. Of course I had backups, encrypted (of course), with a key I had changed recently ... that I knew I needed to save somewhere else, but somehow forgot to. I'm writing this on exactly that laptop, restored completely from the "unreadable" SSD. But it took me some time to read the relevant kernel code, develop and run some helpful tools to search and decrypt the data on said partition. What I have found: It is unlikely that hibernation is the cause to the problem I have encountered. It is just the trigger. Somehow dm-integrity or dm-crypt manages to fuck up it's on-disk meta-data. (Meanwhile the same happened to my work desktop, which had a similar setup, after suspend to RAM.) After I had determined the exact encryption algorithm and layout of my data, I was able to not only read all of it, but the on-disk integrity-tags matched 100%. Every single sector. BR
On Mon, 6 Apr 2020 at 17:37, Markus Schaaf via arch-general <arch-general@archlinux.org> wrote:
What I have found: It is unlikely that hibernation is the cause to the problem I have encountered. It is just the trigger. Somehow dm-integrity or dm-crypt manages to fuck up it's on-disk meta-data. (Meanwhile the same happened to my work desktop, which had a similar setup, after suspend to RAM.) After I had determined the exact encryption algorithm and layout of my data, I was able to not only read all of it, but the on-disk integrity-tags matched 100%. Every single sector.
That's very interesting, and well done for figuring it out and saving the system. Strangely, my problem just went away: my laptop's been hibernating fine since about a week after I started the thread. I put it down to a kernel update. That being said, I don't hibernate regularly, so maybe I've just been lucky :p Paul
Am 23.04.20 um 10:14 schrieb Paul Dann via arch-general:
it down to a kernel update. That being said, I don't hibernate regularly, so maybe I've just been lucky :p
Probably. I needed to read some of the code and it hasn't changed lately AFAIK. My guess is that the problem has something to do with the integrity journal. I had it disabled, but it's read on cryptsetup open anyway. After all I didn't like the (inflexible) data layout (which leads to write amplification), and the journal (and perhaps some of the code) of dm-integrity, so I've formatted all disks back to aes-xts. :-) And remember to print your backup keys on paper. BR
participants (3)
-
Justin Capella
-
Markus Schaaf
-
Paul Dann