Hot RAID swapping?
Hot RAID swapping?
I'd like to swap my spinning disks with SSD drives. I have the new disks and they're just larger than the old ones. My configuration is a RAID-5 with 3 disks (and one hot spare). Can I hot swap a single disk (HDD to SSD), wait for the new disk to rebuild, then repeat?
I'm thinking that I'd mark down the hot spare, replace it with an SSD, mark the SSD as hot spare, mark HDD 1 as "bad" causing the hot spare to activate, then repeat for the other 2 HDDs. I don't have a lot of experience with RAID, but did perform a single disk swap once with success.
If this is a bad idea, why? What's the best way to upgrade?
I'm not sure if this is the right community for this question. If not, please guide me to the right one.
First thing, back up your data in case something goes south.
With that out of the way, I have done this exact thing many times with both zfs and an array managed by mdadm. Depending on the filesystem on top of your raid array, there may be some additional commands you'll need to run to extend your partition to use all of the available space in your larger array. You do have to wait between each disk swap to wait for the array to rebuild so it might be quicker to just make a backup, remove the old drives, and build a new array with your new drives and then copy the data back over. That being said, it is fun to hot swap all of those disks and not lose any data or have downtime assuming everything goes smoothly.
Yes, it's a bad idea to do it this way. The most likely time a RAID array will fail is during a rebuild as that is a whole bunch of drive activity over a sustained timeframe.
Better to perform a backup or copy, power down, remove all the old drives, install the new ones, power back up, configure a new array (most people recommend to use RAID 6 at a minimum, no hot spare, so you have two drive redundancy) then restore or copy back the data.
This way you can also keep the old drives as a cold backup of sorts, potentially reimporting the configuration if needed.
Do you really need the RAID online all the time? Because if you can afford to shut it down for a few hours, it is way less work to do a backup, and then build a new RAID with your SSDs.
I'm not sure if the RAID controller will like two different kind of drives. I'd check the docs if it says something.
I don't really need it online all the time, but I don't expect that I'll find time to do it all at once and I thought swapping may be a way to break up the job into interruptible segments.
You might also want to check that your controller really supports hot-swapping. Not all do.
The safest way is to make a backup and restore it to the new array.
Your way sounds pretty fun though. I'm sure the firmware will complain, but once you have a solid backup you can go nuts!
Also consider a solution such as Bachefs - RAID is cool and all, but this will get you most of the way there too. (Decide on the data safety for yourself but I'm running a 90tb pool with 24tb of metadata / cache on SSD)
The answer to this is going to depend on your raid controller. Frequently the answer is yes, but some raid controllers are more particular about exactly what drives are in the array. Software raid controllers are typically more forgiving of this type of change.
I have a feeling you'd end up with a bunch of big drives with small volumes on them if it did work.
Warning you, I've had issues with RAID combining SSD and HDD. Basically I was on an older dell server and I wanted to do mirroring and the bios straight up refused to do it because it didn't want to mix ssds and hdds.
I suspect I'll have an issue after reading many of the comments. This is also an older Dell server. My only real advantage is that it's currently hosting 2 VMs and one is just a test server, so I don't mind losing the data.
Imaging software is a godsend for that sort of thing. I ended up using BartPE for something like that, and it worked great -- it has a free imaging program on it. You only need a removable drive large enough for all your files since it'll compress everything.
The "Right answer" is copy all the content to another device, upgrade disks, copy content back to new array that can take advantage of the larger disks. Or even safer, set up a completely new device in parallel and then copy data over, decommission the old device.
Some RAID systems will let you do what you say, but will not let you expand the usable storage space. Some will have particular disks they'll even take. It really depends on the RAID system.
Consider moving to RAID-6. Single redundancy is not cool anymore.
This is just a homelab, running on old hardware. I don't think the raid controller supports RAID-6 and I only got 3 new SSDs, and I read that RAID-6 needs 4 drives.
Correct. Get a 4th drive. You will be thankful one day down the road when you are rebuilding the array and you lose a drive during the rebuild.
You've got the right community IMHO. This is something that I've never tackled but I could imagine that it would work. Just make certain you have solid backups just in case the worst should happen.