How to RAID
I know the title kinda sounds weird, but how do I RAID my external harddisk? I'm planning to back-up my data in my ext HD (yes, backing up my back-up lol), and from what I heard, RAIDing the harddisk is one of the best method out there...
I wanted to back it up since I heard it's just a matter of time before a harddisk can go BOOM, and the idea of burning my data to a DVD one at a time just took to long for me...
And since I don't know a bit about computer-related stuff, I guess this is the right place to get any pointer for it.. (Heck, I don't even know whether the method really called RAID or not >.<)
And since we are discussing about HD, I would also like to know what method of backup you comrades prefer? I actually prefer to burn it into a hardcopy (such as DVD/CD), but it's out of the question right now since my data is just kinda too big, and my Superdrive can't read any CD anymore...
Help would be much appreciated!
Is this kind of like if I wanted to "raid" my neighbor's liquor cabbnet?
No, I mean a method of backing up data called RAID.. sry if it confused you, coz I'm actually not sure myself ^^;;;
RAID does not equal backup.
RAID means redundancy array of independence discs: it means to improve performance and reliability of a storage facility.
Backup means to save your data at a storage facility to another which is physically separated from the original.
Backup to DVD-/+R or BD-R is not a really good idea, these media tends to fail over time. And the process might be even faster compared to a HD that has been left alone.
Lastly, there's no way you can RAID external USB HDs. You can buy a pre-built or DIY NAS (network attached storage) with RAID ability though. But a NAS is only needed if you want to access this storage space from multiple devices in your network.
For your need I would recommend buying another external HD and use a backup application that support backup to multiple storage spaces at once. Sorry but I have no idea about a real product that does this though. Others might prove useful here.
For me I simply write a script that call rsync to backup my stuffs to a NAS. In this case if I want a backup of a backup I will write another script and schedule it to copy my data out to a external HD that connects to this NAS.
Really? I originally thought it would be safer if I just burn it into DVDs and just stored it away in a safe place.
And by NAS, do you mean like some kind of stored the data into a 'cloud' storage or something? sorry, totally clueless about this ^^;
http://en.wikipedia.org/wiki/Network-attached_storage
Basically it's an external HD but instead of attaching it using USB/firewire/eSATA then it's connected though ethernet. The disk then contains a small fileserver which means you can mount the disk just like any other disk. You will not notice a noteworthy difference in Finder.
Here is an example: http://www.lacie.com/us/products/product.htm?pid=11250 (I know it's not the cheapest, but it's a quite good one)
Hate to burst your bubble. There is no way you convert your External HDD into a RAID for your system. External HDD are detected as an external storage and there is no OS in the market able to use External HDD as RAID.
However, contrary to the theory that RAID 1 and RAID 5 gives good data redundancy. In true practice, if a RAID is being rebuilt, the I/O will be high; the response of the server is as slow as comming to a halt.
You may opt for a NAS, this is the product I am eyeing currently http://www.lacie.com/asia/products/product.htm?pid=11279 , it basically a file server with RAID1 and the HDD is hot swappable. On top of that it has a bittorrent client preloaded in its firmware, you can download without powering up your PC.
Just my thots.
Well for the same range, I prefer a Synology DS209 http://www.smallnetbuilder.com/nas/nas-reviews/30944?task=view&limit=1 , better functionality and better GUI too.
Great, more product for me to compare :D
The Synology has a near complete feature BT client.
Sure it has a bittorrent client. However it's the worst client I have ever seen. Basically you give it a .torrent and then it downloads everything and you can see how much it has downloaded so far in a primitive display. That's it. I guess it uploads, but the user interface will not tell you. Also the lack of counted uploaded data prevents you from calculating a share radio.
If only you could get radio on each file and the option to pick files in a torrent, then it would be quite decent. Right now it just works in a box of black magic. I guess it's great if you say share a linux CD image permanently (or until it's replaced with a new image) and you don't care how many people who downloads it. It's less good for people who wants to keep track of data going both ways.
Thanks for highlighting the BitTorrent client, I am unaware of its actual details.
Buy a 1TB or greater external HDD and make an image of your original HDD and store the image in the new HDD.
Personally, I would purchase a Windows Home Server to stash all my videos and share them easily over a network. WHS doesn't have RAID however, WHS does have its own data replication system.
Actually you can raid external HDs. However I'm not entirely sure that's what you want to do. You can set up a software raid and it works like this: if you want to write a block, then the OS writes that block to HD1, then HD2 (providing you use mirror/RAID1 mode). Naturally this is slow.
Setup guide: http://support.apple.com/kb/TA24359 (I only know how to do this in OSX, sorry)
The biggest issue with this is that the software mode are far from perfect. The HDs have a tendency to go out of sync and then HD1 needs to be copies to HD2 (or the other way). The computer can do this on it's own, but it's a process which takes 20-30 hours and if you stop before it's done, then you have to start over. Also while they are out of sync (also known as degraded RAID), then the protection is no good.
A raid on a NAS have already been mentioned. I have a Lacie NAS running RAID1. It works perfectly, though it's not the cheapest solution. It's not the fastest either. I benchmarked it to around 12 MB/s. I benchmarked the HD on the other computer to 60 MB/s, just for comparison. While I didn't expect blasting high speed I'm still a bit disappointed with this speed. Oh well it's just for storage of video and pictures being shared by 2 computers and it's fast enough for that. (ok, it's used for other stuff as well since it's there anyway)
It's not true that you can't get hardware raid on USB. Look here http://www.lacie.com/us/products/product.htm?pid=11140
I would highly recommend eSATA though. It's way faster and it's the only interface for external HDs, which supports S.M.A.R.T. (not sure if that is that important for a raid1 though)
You can read S.M.A.R.T information from an external HD, too. This depends on the SATA to USB controller but among the external HDs I have checked I can check all of them.
About what the OP wants, he wants another external HD and RAID it with the one he currently has. From your information I guess you can do it with OS but it's highly unstable at best. And as I said data redundancy does not equal backup. Rebuilding a degraded RAID system is a PITA. So a simple script with another new external HD is the way to go IMO.
In case he wants to use the storage for his entire network, a NAS is the way to go. http://www.smallnetbuilder.com/nas/nas-reviews/30944?task=view&limit=1 This chart can show him what he needs to buy if he wants a NAS and then use the current external HD to backup the NAS.
It's quite interesting that you claim S.M.A.R.T. to be supported by external drives. Last time I checked the official statement was that no USB/Firewire chips supported it and I haven't seen any drives at all, which supports it. If you have one, which does, then I would like to know which one it is ^^
If you want a name this is mine http://www.transcendusa.com/products/ModDetail.asp?ModNo=204&LangNo=0&Func1No=&Func2No=
Not sure which version of smartctl is needed, I have used CentOS 5.2 previously, the default smartctl unable to read the SMART via the external HDD.
On windows HDDScan reads it just fine. Only Linux I haven't tried smartctl to read S.M.A.R.T value from an external HD.
@tymmur_ : I think SMART is only good to detect earlier failure for proactive replacement of the HDD. RAID 1 card which is having ability SMART will will a warning for HDD replacement before the HDD really fails. But if the RAID 1 is not hot-swappable, I think it would defeat the purpose for enterprise solution.
From my understanding, most enterprise servers I encountered so far run RAID 10 or some call it as a mirrored RAID 5. Not sure how useful is it if it is not hot swappable.
Unfortunately, at the moment, lack of finance to support really state of the art multi-million dollar servers/equipments.
Good point. If you use RAID1 internally then the need for S.M.A.R.T. is minimal. Still it's better than nothing and people should be aware of this. It could give an early warning if a drive is worn and might not spin up in a week. It will not warn against sudden failure, like a big dog ramming the computer and the drive makes a headcrash. Nothing is perfect ^^
It's all good, since I use OSX anyway, it's just that I still need time to organize all of the information that I got from reading this thread... lots of terminology that I don't even know existed ^^;;;
If you use OSX 10.5 (or newer) then the easy and recommended solution is to get an external HD and then use it for time machine. Here is how you do it.
Get an external HD, which is bigger than your system partition (read: your internal HD).
Format it with HFS+ as one big partition.
Maybe OSX will ask if you want to use that one for time machine. If you say yes, then that's it and you are done.
If it fails to ask, then open system prefs and then time machine. Select the external HD as time machine target and enable.
The system will then make a backup of everything on your system partition every hour and if your internal HD dies, replace it, install OSX normally and when you get to setting up users, select "restore from time machine" and it will restore everything.
You can also use time machine to restore wrongly deleted files, backup more than one disk or mark a folder to not being backed up (useful for say not copying the live buffer from EyeTV. It can be several GB of buffered video).
I highly recommend getting a disk with Firewire 800 and eSATA support, even if you do not need it right now. It's MUCH faster. Firewire has a superior protocol for file copying. This means that in real life situations firewire 400 (400 Mbit/s) is actually noteworthy faster than USB2 (480 Mbit/s).
Say you are really unlucky and a visitor spills a liquid into your computer (it's always friends or siblings, who does that, never yourself :P) and it fries, then you get a new mac and you just connect it to the external HD and restore everything. You don't have to restore on the computer which made the backup.
Hmmm, but I heard TM took a long time to boot? I also heard an app called Carbon Copy Cloner. Do u have any experience regarding it? I read from forums that it takes a whole less time to boot compared to TM
Just want to add more information here. One of the simplest solution is to make partition on your hard disk (or external HD). When your hard disk fails, usually it only happen on one partition. Meaning that if you have two partition and your hard disk goes boom, high chance that only one of them is dead. It simpler and cheaper than getting RAID. But of course if you have a bad luck, there will be times when every partition fail.
Good strategy on a single Drive. However in actual practice it is best to have another physical HDD for the backup. Also contrary to popular believe, RAID 1 do not really work like a magic bullet. For the RAID rebuilding can take ages. Perhaps with SSD comes into the picture, the RAID rebuilding would be sped up. Personally, I prefer to run on RAID 0 with a barebone backup running offline. However, such solution add more cost.
That's why I have two backups. One in the same HDD but different partition, and the other one is in external HDD ( also partitioned ). I put a different schedule on two of them to reduce the backup time.
For personal use, this should be more than enough.
Damn right man :) More backup the better, the only thing restraining us is the moolah factor.
Actually using two partitions aren't the right way to go. The reason is that it takes very little to kill all partitions. One thing could be a corrupted partition table. There is also the risk of mechanical failures. I have encountered a drive, which failed to spin up and I know somebody else who tried the same thing.
SSD might be safer to use with two partitions, but still not as safe as having two drives. I read about a guy who used RAID1 to be safe and the raid controller (hardware) failed. That turned out to be a real problem because the raid controller was discontinued and without a controller 100% like the one which broke he couldn't access the drives.
The best solution is to have backup on multiple disks and ensure that each disk will continue to work even if another one fails. Simple HDs today aren't that expensive. The price shows up if you want fast drives, but if it's just a script in the background then speed aren't much of an issue unless you need to restore your copy.
Indeed, RAID would save a ton of work on my part as I have to constantly back up my stuff.
I personally back everything that's in my External HD into another 1TB HD, thus always ensuring that I have 2 of anything at one time. The downside is that the copy and pasting is exceedingly tedious.
If your OS is linux based, using of rsync command will simplify the process :D
If it's linux based, then it can be scripted and added as a cronjob. That way you will not even have to care about making the backup. It all handles itself on it's own. It's almost like Time Machine on OSX (though TM has file history and a nice GUI for restoring a backup)
Ditto ^_^ Thank god for Linux.
I know some people worship Linus Torvalds, but calling him God is likely overdoing it :P
Not really referring Linus as God, in a sense thanking God of allowing Linus to come to exist in which leads to formation of linux as we know today ;)
The same thing can be done with Windows http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp . Well you can collect each of the pieces on your own but this one is good to do it quick.
@duo2 : Sorry if I have hijacked your post, no bad intention. Best way to backup your date is using a portable HDD :D
It's okay. Actually, this post gave me so much information that it actually gives me a hard time to understand ^^; I'm still reading it very very slowly to get the idea of it ^^;;;;
That's the risk you will have to take when asking online. Specially computer questions tend to generate a lot of replies everywhere online ^^
You can always ask if there is something you don't get. However some of the answers are already decent and it can be a bit tricky to make it more clear unless you do a bit of work to understand how this field of computers work.
As for hijacking. I don't see any hijacking here. A user hijacking is a user, who goes "oh a computer question. I have this totally unrelated computer question and now everybody should talk about my problem". Nobody did that here as everything is about backup, raid and external drives. All comments (more or less) contains useful information relating to the original question.
Ha, backup is for wimps. I use suicide RAID 0 on all my machines. More speed, more space, and the nice gambling feeling of potentially losing all my stuff when one drive fails.
It's not a matter of if a drive fails, but when. I have personally lost 5 drives >_<
Sure one could argue that after 7 years a drive is worn out, but one drive died after just 2 weeks and 2 within the first year. I don't trust HDs to work tomorrow and if I use RAID0 (I might due to speed) then it's not without a backup system.
best bet in mac os x? like previously said, use time machine against another hdd..... or if u really really want to be safe, build a lime-tech unraid box (or just get a nas), fibre internet, and stash the unraid/nas in another building.... thats what i do...