Failed NVMe M.2 SSD, broken filesystem, unwriteable; can I wipe it anyway?
1
vote
2
answers
3917
views
My Samsung 970 EVO M.2 500GB SSD (MZ-V7E500BW) suddenly failed yesterday during a power outage.
I now have a warning during POST ("WARNING! Please back up your data and replace your hard disk drive.
WARNING! Your HDD/SSD might crash at any moment."). The last time I rebooted before this was about 5 days earlier, and the warning was not present then.
By booting a live USB stick I managed to check the SMART log:
Smart Log for NVME device:nvme0 namespace-id:ffffffff
critical_warning : 0x8
temperature : 49 C
available_spare : 29%
available_spare_threshold : 10%
percentage_used : 0%
endurance group critical warning summary: 0
data_units_read : 4,948,748
data_units_written : 20,573,476
host_read_commands : 100,316,217
host_write_commands : 357,643,056
controller_busy_time : 1,790
power_cycles : 24
power_on_hours : 4,570
unsafe_shutdowns : 11
media_errors : 41
num_err_log_entries : 70
Warning Temperature Time : 0
Critical Composite Temperature Time : 0
Temperature Sensor 1 : 49 C
Temperature Sensor 2 : 74 C
Thermal Management T1 Trans Count : 0
Thermal Management T2 Trans Count : 0
Thermal Management T1 Total Time : 0
Thermal Management T2 Total Time : 0
Messages from the kernel mentioning
nvme
during startup of the live USB OS:
Oct 26 19:18:58 ubuntu kernel: [ 1.233479] nvme nvme0: pci function 0000:06:00.0
Oct 26 19:18:58 ubuntu kernel: [ 1.243303] nvme nvme0: missing or invalid SUBNQN field.
Oct 26 19:18:58 ubuntu kernel: [ 1.243323] nvme nvme0: Shutdown timeout set to 8 seconds
Oct 26 19:18:58 ubuntu kernel: [ 1.252449] nvme nvme0: 4/0/0 default/read/poll queues
Oct 26 19:18:58 ubuntu kernel: [ 1.254855] nvme0n1: p1 p2 p3
Oct 26 19:18:58 ubuntu kernel: [ 3.629244] EXT4-fs (nvme0n1p2): INFO: recovery required on readonly filesystem
Oct 26 19:18:58 ubuntu kernel: [ 3.629246] EXT4-fs (nvme0n1p2): write access will be enabled during recovery
Oct 26 19:18:58 ubuntu kernel: [ 3.674861] blk_update_request: critical medium error, dev nvme0n1, sector 124928 op 0x1:(WRITE) flags 0x800 phys_seg 4 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.674893] Buffer I/O error on dev nvme0n1p2, logical block 0, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.674913] Buffer I/O error on dev nvme0n1p2, logical block 1, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.674931] Buffer I/O error on dev nvme0n1p2, logical block 2, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.674949] Buffer I/O error on dev nvme0n1p2, logical block 3, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.674967] blk_update_request: critical medium error, dev nvme0n1, sector 133200 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.674995] Buffer I/O error on dev nvme0n1p2, logical block 1034, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.675013] blk_update_request: critical medium error, dev nvme0n1, sector 133384 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.675040] Buffer I/O error on dev nvme0n1p2, logical block 1057, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.675059] blk_update_request: critical medium error, dev nvme0n1, sector 147176 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.675086] Buffer I/O error on dev nvme0n1p2, logical block 2781, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.675105] blk_update_request: critical medium error, dev nvme0n1, sector 4319360 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.675132] Buffer I/O error on dev nvme0n1p2, logical block 524304, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.675151] blk_update_request: critical medium error, dev nvme0n1, sector 4319488 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.675178] Buffer I/O error on dev nvme0n1p2, logical block 524320, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.675197] blk_update_request: critical medium error, dev nvme0n1, sector 4319544 op 0x1:(WRITE) flags 0x800 phys_seg 2 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.675224] Buffer I/O error on dev nvme0n1p2, logical block 524327, lost async page write
Oct 26 19:18:58 ubuntu kernel: [ 3.675243] blk_update_request: critical medium error, dev nvme0n1, sector 4319816 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.675270] blk_update_request: critical medium error, dev nvme0n1, sector 4320256 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.675297] blk_update_request: critical medium error, dev nvme0n1, sector 4320936 op 0x1:(WRITE) flags 0x800 phys_seg 1 prio class 0
Oct 26 19:18:58 ubuntu kernel: [ 3.729319] EXT4-fs (nvme0n1p2): error loading journal
Oct 26 19:18:58 ubuntu kernel: [ 3.743157] EXT4-fs (nvme0n1p3): INFO: recovery required on readonly filesystem
Oct 26 19:18:58 ubuntu kernel: [ 3.743158] EXT4-fs (nvme0n1p3): write access will be enabled during recovery
Oct 26 19:18:58 ubuntu kernel: [ 3.806113] EXT4-fs (nvme0n1p3): error loading journal
Oct 26 19:19:04 ubuntu kernel: [ 30.724414] blk_update_request: critical medium error, dev nvme0n1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Oct 26 19:19:04 ubuntu kernel: [ 30.752254] blk_update_request: critical medium error, dev nvme0n1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Oct 26 19:19:05 ubuntu kernel: [ 31.346630] blk_update_request: critical medium error, dev nvme0n1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Oct 26 19:19:05 ubuntu kernel: [ 31.365831] blk_update_request: critical medium error, dev nvme0n1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Oct 26 19:19:29 ubuntu kernel: [ 55.502099] blk_update_request: critical medium error, dev nvme0n1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Oct 26 19:19:29 ubuntu kernel: [ 55.516704] blk_update_request: critical medium error, dev nvme0n1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Oct 26 19:24:44 ubuntu kernel: [ 370.116101] blk_update_request: critical medium error, dev nvme0n1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Oct 26 19:24:44 ubuntu kernel: [ 370.130330] blk_update_request: critical medium error, dev nvme0n1, sector 0 op 0x1:(WRITE) flags 0x800 phys_seg 0 prio class 0
Thanks to ddrescue
I managed to clone all of its partitions to a different machine over the network. There were IO errors while extracting both ext4 partitions but with enough retries it eventually got everything.
After that I was able to run e2fsck
on the images, which appeared to succeed, and now I can mount them as read-only loop devices. Data appears to be intact.
I suppose the first question is **is there anything I can do to fix whatever the problem is, and keep using this drive?** I'm assuming not, but I'm definitely open to suggestions.
If I try to run fsck
on one of the partitions from the live USB, this is what happens. I tried all combinations of answers to the questions as you'll see below. I can't understand enough of the manual pages and don't know enough about filesystems or drives to know what options, if any, might help me.
ubuntu@ubuntu:~$ sudo fsck /dev/nvme0n1p3
fsck from util-linux 2.36.1
e2fsck 1.46.3 (27-Jul-2021)
/dev/nvme0n1p3: recovering journal
Superblock needs_recovery flag is clear, but journal has data.
Run journal anyway? yes
fsck.ext4: Input/output error while recovering journal of /dev/nvme0n1p3
fsck.ext4: unable to set superblock flags on /dev/nvme0n1p3
/dev/nvme0n1p3: ********** WARNING: Filesystem still has errors **********
ubuntu@ubuntu:~$ sudo fsck /dev/nvme0n1p3
fsck from util-linux 2.36.1
e2fsck 1.46.3 (27-Jul-2021)
/dev/nvme0n1p3: recovering journal
Superblock needs_recovery flag is clear, but journal has data.
Run journal anyway? no
Clear journal? no
fsck.ext4: Input/output error while recovering journal of /dev/nvme0n1p3
fsck.ext4: unable to set superblock flags on /dev/nvme0n1p3
/dev/nvme0n1p3: ********** WARNING: Filesystem still has errors **********
ubuntu@ubuntu:~$ sudo fsck /dev/nvme0n1p3
fsck from util-linux 2.36.1
e2fsck 1.46.3 (27-Jul-2021)
/dev/nvme0n1p3: recovering journal
Superblock needs_recovery flag is clear, but journal has data.
Run journal anyway? no
Clear journal? yes
fsck.ext4: Input/output error while recovering journal of /dev/nvme0n1p3
fsck.ext4: unable to set superblock flags on /dev/nvme0n1p3
/dev/nvme0n1p3: ********** WARNING: Filesystem still has errors **********
ubuntu@ubuntu:~$
I believe the drive is still under warranty, and I'm trying to get in contact with Samsung support to try to get a replacement or refund.
If they ask me to send it back, that's going to pose a problem since there's sensitive data on this drive.
The drive resists all attempts to write to it. I can't mount it and write to it normally. The kernel emits IO errors if I try to write to it at the block level. Even Samsung's secure erase tool (their Windows-only software offers to produce a bootable USB drive with such a tool) fails.
**Is there some way to force secure erasure of this device?**
Asked by tremby
(563 rep)
Oct 26, 2021, 07:52 PM
Last activity: Apr 30, 2025, 05:03 PM
Last activity: Apr 30, 2025, 05:03 PM