[arch-general] blk_update_request: I/O error, dev sda
Hi Arch is occasionally - only occasionally - very slow to start up. After examining the dmesg log, I see errors like 100.740566] blk_update_request: I/O error, dev sda, sector 119400712 op 0x0:(READ) flags 0x80700 phys_seg 3 prio class 0 [ 100.740584] ata1: EH complete [ 101.100769] audit: type=1130 audit(1622446429.249:86): pid=1 uid=0 auid=4294967295 ses=4294967295 msg='unit=upower comm="systemd" ex e="/usr/lib/systemd/systemd" hostname=? addr=? terminal=? res=success' [ 101.180724] ata1.00: exception Emask 0x0 SAct 0x40 SErr 0x0 action 0x6 [ 101.180730] ata1.00: irq_stat 0x40000008 [ 101.180733] ata1.00: failed command: READ FPDMA QUEUED [ 101.180735] ata1.00: cmd 60/70:30:f8:37:1e/00:00:07:00:00/40 tag 6 ncq dma 57344 in res 41/84:70:fc:37:1e/00:00:07:00:00/00 Emask 0x410 (ATA bus error) <F> [ 101.180742] ata1.00: status: { DRDY ERR } [ 101.180743] ata1.00: error: { ICRC ABRT } [ 101.180747] ata1: hard resetting link [ 101.488857] ata1: SATA link up 1.5 Gbps (SStatus 113 SControl 310) [ 101.594776] ata1.00: configured for UDMA/66 [ 101.594806] sd 0:0:0:0: [sda] tag#6 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_SENSE cmd_age=0s [ 101.594811] sd 0:0:0:0: [sda] tag#6 Sense Key : Aborted Command [current] [ 101.594816] sd 0:0:0:0: [sda] tag#6 Add. Sense: Scsi parity error [ 101.594819] sd 0:0:0:0: [sda] tag#6 CDB: Read(10) 28 00 07 1e 37 f8 00 00 70 00 [ 101.594821] blk_update_request: I/O error, dev sda, sector 119420920 op 0x0:(READ) flags 0x80700 phys_seg 10 prio class 0 [ 101.594844] ata1: EH complete [ 101.870769] ata1.00: exception Emask 0x0 SAct 0x3c SErr 0x0 action 0x6 [ 101.870775] ata1.00: irq_stat 0x40000008 [ 101.870778] ata1.00: failed command: READ FPDMA QUEUED [ 101.870780] ata1.00: cmd 60/b8:18:c0:62:89/00:00:07:00:00/40 tag 3 ncq dma 94208 in res 41/84:b8:48:63:89/00:00:07:00:00/00 Emask 0x410 (ATA bus error) <F> [ 101.870787] ata1.00: status: { DRDY ERR } [ 101.870789] ata1.00: error: { ICRC ABRT } This suggests a hard drive that should be replaced, but badblocks reports that there are absolutely zero bad sectors or other errors on the hard drive. And the fact that the problem is intermittent also suggests that it may perhaps be a kernel issue rather than a hard drive issue? It is not a very critical machine and I have backups. Do you have some suggestions as to how I should go about dealing with this? Other commands to test the hard drive? Kernel parameters? Anything? ~/ % uname -a Linux gatsby 5.12.7-zen1-1-zen #1 ZEN SMP PREEMPT Wed, 26 May 2021 22:03:55 +0000 x86_64 GNU/Linux Thanks, Morten
On Mon, May 31, 2021 at 01:11:19PM +0200, Morten Bo Johansen via arch-general wrote:
Do you have some suggestions as to how I should go about dealing with this? Other commands to test the hard drive? Kernel parameters? Anything?
Check if it's a bad sata cable. These errors usually indicates a broken cable, connector or a bad disk. But if there are no bad sectors I wouldn't bet on the disk being broken.
Thanks, Morten
Clearly we can only resolve this naming conflict one way: I demand holmgang! -- Morten Linderud PGP: 9C02FF419FECBE16 For the uncivilized among us: https://en.wikipedia.org/wiki/Holmgang
On 2021-05-31 Morten Linderud via arch-general wrote:
Check if it's a bad sata cable. These errors usually indicates a broken cable, connector or a bad disk. But if there are no bad sectors I wouldn't bet on the disk being broken.
I will check the cable.
Clearly we can only resolve this naming conflict one way:
I demand holmgang!
I'll give it to you, unless you are bigger than me, of course - I only fight men who are smaller than me and who pose no threat ;) Thanks, Morten
On 31.05.21 13:11, Morten Bo Johansen via arch-general wrote:
Do you have some suggestions as to how I should go about dealing with this? Other commands to test the hard drive? Kernel parameters? Anything?
The output of `smartctl -x /dev/sda` could provide some further insights into the health state of that drive. Cheers -- Thore "foxxx0" Bödecker GPG ID: 0xEB763B4E9DB887A6 GPG FP: 051E AD6A 6155 389D 69DA 02E5 EB76 3B4E 9DB8 87A6
On 2021-05-31 Thore Bödecker via arch-general wrote:
The output of `smartctl -x /dev/sda` could provide some further insights into the health state of that drive.
Good call. It reports that there are 12 errors on the drive. They all look more or less the same: Error 9 [8] occurred at disk power-on lifetime: 19635 hours (818 days + 3 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 09 c0 0b 06 00 00 Error: UNC at LBA = 0x09c00b06 = 163580678 I suppose I should first replace the cable and if the problem persists then replace the drive. Thanks, Morten
On 31.05.21 13:37, Morten Bo Johansen via arch-general wrote:
On 2021-05-31 Thore Bödecker via arch-general wrote:
The output of `smartctl -x /dev/sda` could provide some further insights into the health state of that drive.
Good call. It reports that there are 12 errors on the drive. They all look more or less the same:
Error 9 [8] occurred at disk power-on lifetime: 19635 hours (818 days + 3 hours) When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 09 c0 0b 06 00 00 Error: UNC at LBA = 0x09c00b06 = 163580678
I suppose I should first replace the cable and if the problem persists then replace the drive.
Uncorrectable Errors with specific LBAs are usually a very solid indicator for a defective drive. If you could share the full `smartctl -x /dev/sda` output, we might be able to establish/identify some further hints on a failing drive instead of a bad cable. I would strongly suggest to take a final backup of whatever is still accessible (if desired) and then decomission the drive. Cheers -- Thore "foxxx0" Bödecker GPG ID: 0xEB763B4E9DB887A6 GPG FP: 051E AD6A 6155 389D 69DA 02E5 EB76 3B4E 9DB8 87A6
On 2021-05-31 Thore Bödecker via arch-general wrote:
Uncorrectable Errors with specific LBAs are usually a very solid indicator for a defective drive.
If you could share the full `smartctl -x /dev/sda` output, we might be able to establish/identify some further hints on a failing drive instead of a bad cable.
I include it below. It says "Device Error Count: 12". Oddly enough, the command sudo smartctl -t long /dev/sda completes without any errors at all. The problem is also intermittent. It is rather odd. Thanks, Morten --------------- output from sudo smartctl -x /dev/sda ----------------- smartctl 7.2 2020-12-30 r5155 [x86_64-linux-5.12.7-zen1-1-zen] (local build) Copyright (C) 2002-20, Bruce Allen, Christian Franke, www.smartmontools.org === START OF INFORMATION SECTION === Model Family: Seagate Laptop SSHD Device Model: ST500LM000-1EJ162 Serial Number: W371R17A LU WWN Device Id: 5 000c50 06e8a3600 Firmware Version: LVD5 User Capacity: 500.107.862.016 bytes [500 GB] Sector Sizes: 512 bytes logical, 4096 bytes physical Rotation Rate: 5400 rpm Form Factor: 2.5 inches Device is: In smartctl database [for details use: -P show] ATA Version is: ACS-2, ACS-3 T13/2161-D revision 3b SATA Version is: SATA 3.1, 6.0 Gb/s (current: 1.5 Gb/s) Local Time is: Mon May 31 20:22:01 2021 CEST SMART support is: Available - device has SMART capability. SMART support is: Enabled AAM feature is: Unavailable APM feature is: Disabled Rd look-ahead is: Enabled Write cache is: Enabled DSN feature is: Unavailable ATA Security is: Disabled, NOT FROZEN [SEC1] Wt Cache Reorder: Unavailable === START OF READ SMART DATA SECTION === SMART overall-health self-assessment test result: PASSED See vendor-specific Attribute list for marginal Attributes. General SMART Values: Offline data collection status: (0x00) Offline data collection activity was never started. Auto Offline Data Collection: Disabled. Self-test execution status: ( 0) The previous self-test routine completed without error or no self-test has ever been run. Total time to complete Offline data collection: ( 139) seconds. Offline data collection capabilities: (0x73) SMART execute Offline immediate. Auto Offline data collection on/off support. Suspend Offline collection upon new command. No Offline surface scan supported. Self-test supported. Conveyance Self-test supported. Selective Self-test supported. SMART capabilities: (0x0003) Saves SMART data before entering power-saving mode. Supports SMART auto save timer. Error logging capability: (0x01) Error logging supported. General Purpose Logging supported. Short self-test routine recommended polling time: ( 1) minutes. Extended self-test routine recommended polling time: ( 96) minutes. Conveyance self-test routine recommended polling time: ( 2) minutes. SCT capabilities: (0x1081) SCT Status supported. SMART Attributes Data Structure revision number: 10 Vendor Specific SMART Attributes with Thresholds: ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE 1 Raw_Read_Error_Rate POSR-- 118 099 006 - 192553232 3 Spin_Up_Time PO---- 099 098 000 - 0 4 Start_Stop_Count -O--CK 098 098 020 - 2850 5 Reallocated_Sector_Ct PO--CK 100 100 010 - 0 7 Seek_Error_Rate POSR-- 082 060 030 - 4492926169 9 Power_On_Hours -O--CK 078 078 000 - 19702 10 Spin_Retry_Count PO--C- 100 100 097 - 0 12 Power_Cycle_Count -O--CK 097 097 020 - 3079 184 End-to-End_Error -O--CK 077 077 099 NOW 23 187 Reported_Uncorrect -O--CK 099 099 000 - 1 188 Command_Timeout -O--CK 100 099 000 - 12 189 High_Fly_Writes -O-RCK 067 067 000 - 33 190 Airflow_Temperature_Cel -O---K 051 042 045 Past 49 (Min/Max 28/55 #251) 191 G-Sense_Error_Rate -O--CK 100 100 000 - 0 192 Power-Off_Retract_Count -O--CK 100 100 000 - 169 193 Load_Cycle_Count -O--CK 099 099 000 - 2892 194 Temperature_Celsius -O---K 049 058 000 - 49 (0 9 0 0 0) 197 Current_Pending_Sector -O--C- 100 100 000 - 0 198 Offline_Uncorrectable ----C- 100 100 000 - 0 199 UDMA_CRC_Error_Count -OSRCK 200 200 000 - 0 254 Free_Fall_Sensor -O--CK 100 100 000 - 0 ||||||_ K auto-keep |||||__ C event count ||||___ R error rate |||____ S speed/performance ||_____ O updated online |______ P prefailure warning General Purpose Log Directory Version 1 SMART Log Directory Version 1 [multi-sector log support] Address Access R/W Size Description 0x00 GPL,SL R/O 1 Log Directory 0x01 SL R/O 1 Summary SMART error log 0x02 SL R/O 5 Comprehensive SMART error log 0x03 GPL R/O 5 Ext. Comprehensive SMART error log 0x06 SL R/O 1 SMART self-test log 0x07 GPL R/O 1 Extended self-test log 0x09 SL R/W 1 Selective self-test log 0x10 GPL R/O 1 NCQ Command Error log 0x11 GPL R/O 1 SATA Phy Event Counters log 0x21 GPL R/O 1 Write stream error log 0x22 GPL R/O 1 Read stream error log 0x24 GPL R/O 1223 Current Device Internal Status Data log 0x25 GPL R/O 1223 Saved Device Internal Status Data log 0x30 GPL,SL R/O 9 IDENTIFY DEVICE data log 0x80-0x9f GPL,SL R/W 16 Host vendor specific log 0xa1 GPL,SL VS 20 Device vendor specific log 0xa2 GPL VS 3900 Device vendor specific log 0xa8 GPL,SL VS 129 Device vendor specific log 0xa9 GPL,SL VS 1 Device vendor specific log 0xab GPL VS 1 Device vendor specific log 0xae GPL VS 1 Device vendor specific log 0xb0 GPL VS 4580 Device vendor specific log 0xbe-0xbf GPL VS 65535 Device vendor specific log 0xc1 GPL,SL VS 10 Device vendor specific log 0xc2 GPL,SL VS 50 Device vendor specific log 0xc4 GPL,SL VS 5 Device vendor specific log 0xe0 GPL,SL R/W 1 SCT Command/Status 0xe1 GPL,SL R/W 1 SCT Data Transfer SMART Extended Comprehensive Error Log Version: 1 (5 sectors) Device Error Count: 12 CR = Command Register FEATR = Features Register COUNT = Count (was: Sector Count) Register LBA_48 = Upper bytes of LBA High/Mid/Low Registers ] ATA-8 LH = LBA High (was: Cylinder High) Register ] LBA LM = LBA Mid (was: Cylinder Low) Register ] Register LL = LBA Low (was: Sector Number) Register ] DV = Device (was: Device/Head) Register DC = Device Control Register ER = Error register ST = Status register Powered_Up_Time is measured from power on, and printed as DDd+hh:mm:SS.sss where DD=days, hh=hours, mm=minutes, SS=sec, and sss=millisec. It "wraps" after 49.710 days. Error 12 [11] occurred at disk power-on lifetime: 19675 hours (819 days + 19 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 00 03 51 48 00 00 Error: UNC at LBA = 0x00035148 = 217416 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 00 90 00 00 00 03 50 e8 40 00 00:00:41.749 READ FPDMA QUEUED 61 00 00 00 08 00 00 1c 44 0f 38 40 00 00:00:41.749 WRITE FPDMA QUEUED ea 00 00 00 00 00 00 00 00 00 00 a0 00 00:00:41.748 FLUSH CACHE EXT 61 00 00 00 08 00 00 00 03 57 b0 40 00 00:00:41.714 WRITE FPDMA QUEUED 61 00 00 00 08 00 00 07 00 44 28 40 00 00:00:41.714 WRITE FPDMA QUEUED Error 11 [10] occurred at disk power-on lifetime: 19675 hours (819 days + 19 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 12 eb 60 30 00 00 Error: UNC at LBA = 0x12eb6030 = 317415472 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 01 00 00 00 12 eb 5f f0 40 00 00:00:38.123 READ FPDMA QUEUED 60 00 00 00 c0 00 00 00 03 78 10 40 00 00:00:38.088 READ FPDMA QUEUED 60 00 00 00 08 00 00 12 eb 68 38 40 00 00:00:38.088 READ FPDMA QUEUED 60 00 00 00 80 00 00 00 03 6d c0 40 00 00:00:38.086 READ FPDMA QUEUED 60 00 00 00 68 00 00 00 03 61 b8 40 00 00:00:38.059 READ FPDMA QUEUED Error 10 [9] occurred at disk power-on lifetime: 19675 hours (819 days + 19 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 12 eb 67 c8 00 00 Error: UNC at LBA = 0x12eb67c8 = 317417416 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 01 00 00 00 12 eb 67 c0 40 00 00:00:29.049 READ FPDMA QUEUED 60 00 00 01 00 00 00 00 03 78 d0 40 00 00:00:29.026 READ FPDMA QUEUED 60 00 00 01 00 00 00 12 eb 51 70 40 00 00:00:29.009 READ FPDMA QUEUED 60 00 00 00 e0 00 00 00 03 56 70 40 00 00:00:28.966 READ FPDMA QUEUED 60 00 00 00 08 00 00 2e 84 79 68 40 00 00:00:28.966 READ FPDMA QUEUED Error 9 [8] occurred at disk power-on lifetime: 19635 hours (818 days + 3 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 09 c0 0b 06 00 00 Error: UNC at LBA = 0x09c00b06 = 163580678 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 01 00 00 00 09 c0 0b 00 40 00 00:01:09.134 READ FPDMA QUEUED 60 00 00 00 08 00 00 09 41 12 f8 40 00 00:01:09.133 READ FPDMA QUEUED 60 00 00 00 08 00 00 0d 84 11 08 40 00 00:01:09.133 READ FPDMA QUEUED 60 00 00 00 08 00 00 1c 04 4f 68 40 00 00:01:09.133 READ FPDMA QUEUED 60 00 00 00 08 00 00 08 c1 18 00 40 00 00:01:09.132 READ FPDMA QUEUED Error 8 [7] occurred at disk power-on lifetime: 19635 hours (818 days + 3 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 07 80 08 70 00 00 Error: UNC at LBA = 0x07800870 = 125831280 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 00 80 00 00 07 80 08 00 40 00 00:00:33.186 READ FPDMA QUEUED 60 00 00 00 10 00 00 07 0b 95 d0 40 00 00:00:33.166 READ FPDMA QUEUED 60 00 00 00 08 00 00 11 85 5c 58 40 00 00:00:33.165 READ FPDMA QUEUED 60 00 00 00 30 00 00 16 c4 fd 00 40 00 00:00:33.099 READ FPDMA QUEUED 60 00 00 00 08 00 00 07 41 15 c8 40 00 00:00:33.099 READ FPDMA QUEUED Error 7 [6] occurred at disk power-on lifetime: 19635 hours (818 days + 3 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 07 40 08 10 00 00 Error: UNC at LBA = 0x07400810 = 121636880 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 00 b8 00 00 00 03 81 78 40 00 00:00:31.548 READ FPDMA QUEUED 60 00 00 00 08 00 00 07 40 08 78 40 00 00:00:31.548 READ FPDMA QUEUED 60 00 00 00 28 00 00 07 40 08 48 40 00 00:00:31.548 READ FPDMA QUEUED 60 00 00 00 10 00 00 07 40 08 28 40 00 00:00:31.547 READ FPDMA QUEUED 60 00 00 00 18 00 00 07 40 08 00 40 00 00:00:31.547 READ FPDMA QUEUED Error 6 [5] occurred at disk power-on lifetime: 19635 hours (818 days + 3 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 07 00 08 42 00 00 Error: UNC at LBA = 0x07000842 = 117442626 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 00 08 00 00 07 40 08 78 40 00 00:00:31.349 READ FPDMA QUEUED 60 00 00 00 28 00 00 07 40 08 48 40 00 00:00:31.348 READ FPDMA QUEUED 60 00 00 00 10 00 00 07 40 08 28 40 00 00:00:31.348 READ FPDMA QUEUED 60 00 00 00 18 00 00 07 40 08 00 40 00 00:00:31.348 READ FPDMA QUEUED 60 00 00 00 20 00 00 07 00 08 58 40 00 00:00:31.348 READ FPDMA QUEUED Error 5 [4] occurred at disk power-on lifetime: 19603 hours (816 days + 19 hours) When the command that caused the error occurred, the device was active or idle. After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 1c 00 09 18 00 00 Error: UNC at LBA = 0x1c000918 = 469764376 Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 01 08 00 00 1c 00 09 00 40 00 00:00:25.048 READ FPDMA QUEUED 60 00 00 00 b0 00 00 07 0d e7 58 40 00 00:00:25.045 READ FPDMA QUEUED 60 00 00 01 00 00 00 07 0d e4 d0 40 00 00:00:25.035 READ FPDMA QUEUED 60 00 00 00 e0 00 00 07 0d e3 68 40 00 00:00:25.034 READ FPDMA QUEUED 60 00 00 01 00 00 00 07 0d e6 58 40 00 00:00:25.034 READ FPDMA QUEUED SMART Extended Self-test Log Version: 1 (1 sectors) Num Test_Description Status Remaining LifeTime(hours) LBA_of_first_error # 1 Extended offline Completed without error 00% 19701 - # 2 Extended offline Completed without error 00% 19346 - # 3 Extended offline Completed without error 00% 9255 - # 4 Short offline Completed without error 00% 9253 - SMART Selective self-test log data structure revision number 1 SPAN MIN_LBA MAX_LBA CURRENT_TEST_STATUS 1 0 0 Not_testing 2 0 0 Not_testing 3 0 0 Not_testing 4 0 0 Not_testing 5 0 0 Not_testing Selective self-test flags (0x0): After scanning selected spans, do NOT read-scan remainder of disk. If Selective self-test is pending on power-up, resume after 0 minute delay. SCT Status Version: 3 SCT Version (vendor specific): 522 (0x020a) Device State: Active (0) Current Temperature: 49 Celsius Power Cycle Min/Max Temperature: 25/55 Celsius Lifetime Min/Max Temperature: 13/58 Celsius Under/Over Temperature Limit Count: 0/5 SCT Data Table command not supported SCT Error Recovery Control command not supported Device Statistics (GP/SMART Log 0x04) not supported Pending Defects log (GP Log 0x0c) not supported SATA Phy Event Counters (GP Log 0x11) ID Size Value Description 0x000a 2 3 Device-to-host register FISes sent due to a COMRESET 0x0001 2 0 Command failed due to ICRC error 0x0003 2 0 R_ERR response for device-to-host data FIS 0x0004 2 0 R_ERR response for host-to-device data FIS 0x0006 2 0 R_ERR response for device-to-host non-data FIS 0x0007 2 0 R_ERR response for host-to-device non-data FIS
On 31.05.21 20:27, Morten Bo Johansen via arch-general wrote:
I include it below. It says "Device Error Count: 12". Oddly enough, the command sudo smartctl -t long /dev/sda completes without any errors at all. The problem is also intermittent. It is rather odd.
This seems odd indeed, I am not sure however what an extended/long smart self test exactly does on an SSHD, does it include the cache-SSD as well? It is my understanding though, that the self tests do not cover all involved components of the drive to the full extent and thus a successful long selftest does not necessarily proof the drive is healthy.
--------------- output from sudo smartctl -x /dev/sda ----------------- ID# ATTRIBUTE_NAME FLAGS VALUE WORST THRESH FAIL RAW_VALUE 1 Raw_Read_Error_Rate POSR-- 118 099 006 - 192553232 3 Spin_Up_Time PO---- 099 098 000 - 0 4 Start_Stop_Count -O--CK 098 098 020 - 2850 5 Reallocated_Sector_Ct PO--CK 100 100 010 - 0 7 Seek_Error_Rate POSR-- 082 060 030 - 4492926169 9 Power_On_Hours -O--CK 078 078 000 - 19702 10 Spin_Retry_Count PO--C- 100 100 097 - 0 12 Power_Cycle_Count -O--CK 097 097 020 - 3079 184 End-to-End_Error -O--CK 077 077 099 NOW 23
This indicates a drive failure, depending on what kind of End-to-End errors are logged it might (rarely) be just a defective cable as was already suggested.
187 Reported_Uncorrect -O--CK 099 099 000 - 1 188 Command_Timeout -O--CK 100 099 000 - 12 189 High_Fly_Writes -O-RCK 067 067 000 - 33 190 Airflow_Temperature_Cel -O---K 051 042 045 Past 49 (Min/Max 28/55 #251)
This indicates the drive is running rather hot, I suppose it is enclosed in a laptop with poor ventilation and there is not much you can do about it? However, if you can, a bit more airflow would be appreciated by the drive. But in case you are going to replace it with an SSD (in contrast to the current SSHD), these tend to create less prolonged heat and should run cooler.
191 G-Sense_Error_Rate -O--CK 100 100 000 - 0 192 Power-Off_Retract_Count -O--CK 100 100 000 - 169 193 Load_Cycle_Count -O--CK 099 099 000 - 2892 194 Temperature_Celsius -O---K 049 058 000 - 49 (0 9 0 0 0) 197 Current_Pending_Sector -O--C- 100 100 000 - 0 198 Offline_Uncorrectable ----C- 100 100 000 - 0 199 UDMA_CRC_Error_Count -OSRCK 200 200 000 - 0 254 Free_Fall_Sensor -O--CK 100 100 000 - 0
Error 12 [11] occurred at disk power-on lifetime: 19675 hours (819 days + 19 hours) When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 00 03 51 48 00 00 Error: UNC at LBA = 0x00035148 = 217416
Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 00 90 00 00 00 03 50 e8 40 00 00:00:41.749 READ FPDMA QUEUED 61 00 00 00 08 00 00 1c 44 0f 38 40 00 00:00:41.749 WRITE FPDMA QUEUED ea 00 00 00 00 00 00 00 00 00 00 a0 00 00:00:41.748 FLUSH CACHE EXT 61 00 00 00 08 00 00 00 03 57 b0 40 00 00:00:41.714 WRITE FPDMA QUEUED 61 00 00 00 08 00 00 07 00 44 28 40 00 00:00:41.714 WRITE FPDMA QUEUED
Error 11 [10] occurred at disk power-on lifetime: 19675 hours (819 days + 19 hours) When the command that caused the error occurred, the device was active or idle.
After command completion occurred, registers were: ER -- ST COUNT LBA_48 LH LM LL DV DC -- -- -- == -- == == == -- -- -- -- -- 40 -- 51 00 00 00 00 12 eb 60 30 00 00 Error: UNC at LBA = 0x12eb6030 = 317415472
Commands leading to the command that caused the error were: CR FEATR COUNT LBA_48 LH LM LL DV DC Powered_Up_Time Command/Feature_Name -- == -- == -- == == == -- -- -- -- -- --------------- -------------------- 60 00 00 01 00 00 00 12 eb 5f f0 40 00 00:00:38.123 READ FPDMA QUEUED 60 00 00 00 c0 00 00 00 03 78 10 40 00 00:00:38.088 READ FPDMA QUEUED 60 00 00 00 08 00 00 12 eb 68 38 40 00 00:00:38.088 READ FPDMA QUEUED 60 00 00 00 80 00 00 00 03 6d c0 40 00 00:00:38.086 READ FPDMA QUEUED 60 00 00 00 68 00 00 00 03 61 b8 40 00 00:00:38.059 READ FPDMA QUEUED
[...]
[...]
Since these errors are not UDMA/CRC errors (which might be caused by a defective cable) and only occured very recently (error timestamp in relation to Power_On_Hours) a faulty cable seems very unlikely if you did not recently poke at it (e.g. reconnected/reseated the drive for whatever reason). In my opinion this drive is on the brink of failure and should be replaced immediately. Cheers -- Thore "foxxx0" Bödecker GPG ID: 0xEB763B4E9DB887A6 GPG FP: 051E AD6A 6155 389D 69DA 02E5 EB76 3B4E 9DB8 87A6
On 2021-05-31 Thore Bödecker via arch-general wrote:
187 Reported_Uncorrect -O--CK 099 099 000 - 1 188 Command_Timeout -O--CK 100 099 000 - 12 189 High_Fly_Writes -O-RCK 067 067 000 - 33 190 Airflow_Temperature_Cel -O---K 051 042 045 Past 49 (Min/Max 28/55 #251)
This indicates the drive is running rather hot, I suppose it is enclosed in a laptop with poor ventilation and there is not much you can do about it? However, if you can, a bit more airflow would be appreciated by the drive. But in case you are going to replace it with an SSD (in contrast to the current SSHD), these tend to create less prolonged heat and should run cooler.
It is a Lenovo M93p ThinkCentre which is indeed not in a very well ventilated place.
Since these errors are not UDMA/CRC errors (which might be caused by a defective cable) and only occured very recently (error timestamp in relation to Power_On_Hours) a faulty cable seems very unlikely if you did not recently poke at it (e.g. reconnected/reseated the drive for whatever reason).
In my opinion this drive is on the brink of failure and should be replaced immediately.
OK, I will heed your advice. Too bad, I just installed Arch a couple of months ago and I have set up a really nice system. Now, I have to go through it all again ;( Thanks for taking the time, Morten
On 31.05.21 22:10, Morten Bo Johansen via arch-general wrote:
OK, I will heed your advice. Too bad, I just installed Arch a couple of months ago and I have set up a really nice system. Now, I have to go through it all again ;(
For a quick and dirty disaster-recovery backup you could try dumping a list of your installed packages, rsync'ing /etc/ along with some config files from your home directory to a flashdrive. Otherwise I would encourage you to take a look at borg [1] for a long-term backup solution with compression and deduplication and a server-delta mode similar to rsync for improved performance on slower links (if your backup storage is offsite). You might also want to dump your partition table / lvm / btrfs layout and bootloader entries/configs. While the drive is not completely wrecked yet, you could use the chance to gather as much as is possible. At least this would give some sort of reference on "how did I do that last time?" things and ease + speed up your install on the next drive. Once you have done that, you could also try migrating your installation from this drive if it is not dead once the new one arrived. Simply install the new one in the internal slot and use some sort of external usb3.0<->sata adapter or enclosure. Then boot archiso, partition + format + mount your new drive, mount your old drive in a separate structure (e.g. /mnt2 as root) and then rsync your data. I would suggest `rsync -phaxsPHAXS` for parameters which has served me well for this kind of purpose, you need to repeat that for each *filesystem* (due to -x which I strongly recommend though) beneath the root (i.e. repeat for /boot, /home, ...). Afterwards make sure to correct the filesystem UUIDs in your /etc/fstab, reinstall the boot loader, correct the filesystem UUIDs in your boot loader entries, unmount everything cleanly, reboot and enjoy. If you have further questions on this procedure, feel free to ask. Cheers [1] https://borgbackup.readthedocs.io/en/stable/ -- Thore "foxxx0" Bödecker GPG ID: 0xEB763B4E9DB887A6 GPG FP: 051E AD6A 6155 389D 69DA 02E5 EB76 3B4E 9DB8 87A6
On Mon, 31 May 2021 at 21:12, Morten Bo Johansen via arch-general <arch-general@lists.archlinux.org> wrote:
badblocks reports that there are absolutely zero bad sectors or other errors on the hard drive.
https://serverfault.com/questions/506281/how-are-smart-selftests-related-to-...
participants (4)
-
David
-
Morten Bo Johansen
-
Morten Linderud
-
Thore Bödecker