RAID5 - Mark a disk faulty during reshape
4
votes
1
answer
329
views
# Context
I have a software RAID5 array (mdadm) on 3 disks. Last week, one disk started to get reading issue:
# dmesg
ata3.00: exception Emask 0x0 SAct 0x30000001 SErr 0x0 action 0x0
ata3.00: irq_stat 0x40000008
ata3.00: failed command: READ FPDMA QUEUED
ata3.00: cmd 60/08:e0:40:0b:c6/00:00:a1:00:00/40 tag 28 ncq dma 4096 in
res 41/40:00:40:0b:c6/00:00:a1:00:00/40 Emask 0x409 (media error)
ata3.00: status: { DRDY ERR }
ata3.00: error: { UNC }
ata3.00: configured for UDMA/133
sd 2:0:0:0: [sdc] tag#28 FAILED Result: hostbyte=DID_OK driverbyte=DRIVER_OK cmd_age=5s
sd 2:0:0:0: [sdc] tag#28 Sense Key : Medium Error [current]
sd 2:0:0:0: [sdc] tag#28 Add. Sense: Unrecovered read error - auto reallocate failed
sd 2:0:0:0: [sdc] tag#28 CDB: Read(16) 88 00 00 00 00 00 a1 c6 0b 40 00 00 00 08 00 00
I/O error, dev sdc, sector 2714110784 op 0x0:(READ) flags 0x0 phys_seg 1 prio class 2
ata3: EH complete
So I've formatted and added a new device to the array and then grow the array
# mdadm --add /dev/md0 /dev/sdd1
# mdadm --grow --raid-devices=4 /dev/md0
It seems that it wasn't the best idea. Due to reading issue of the faulty disk, reshape operation estimated duration is more or less 6 months. (below 12-hours progress)
$ cat /proc/mdstat
Personalities : [raid6] [raid5] [raid4] [linear] [multipath] [raid0] [raid1] [raid10]
md0 : active raid5 sdd1 sdc1 sde1 sdb1
5860269184 blocks super 1.2 level 5, 64k chunk, algorithm 2 [4/4] [UUUU]
[>....................] reshape = 0.2% (6232960/2930134592) finish=265471.5min speed=183K/sec
bitmap: 4/22 pages [16KB], 65536KB chunk
unused devices:
So many events can occur meanwhile like power issue or second disk fail for example.
I would love telling mdadm to stop reading the faulty disk but it seems stopping reshaping operation may lead to data loss.
# Questions
1. Should I mark faulty the disk with reading issue while reshaping operation?
2. Is there a clever way to speed up reshaping?
3. Any other advices?
Thanks a lot for your ideas and help.
Asked by BiBzz
(43 rep)
Jun 17, 2024, 11:57 AM
Last activity: Jun 17, 2024, 02:55 PM
Last activity: Jun 17, 2024, 02:55 PM