Folks - this may or not be OT...

I had a five drive RAID 1+0 (software RAID) which faied on me. As it happens, one drive failed and the software began writing to the 5th drive (hot spare). Unfortunately, during this time I got a BSOD event and a second drive failed. Then I was really stuck and couldn't even boot from a CD. Needless to say, I had to call in a data recovery company and God knows what that's going to cost - but it sounds like they've recovered everything.

What I need now is a bullet proof recovery/backup strategy. So if anyone out there has suggestions, please let me know.

To perhaps give you an idea, I was thinking of two main options:

1. A hardware RAID controller (PCI card) and two eSATA drives for regular backups
2. Software RAID as before with 6 drives plus a NAS box for backups

Any backup software needs to be simple and completely automatic. I just want to set it up once and then just leave it. So please don't tell me to get a large USB drive and remember to backup to it...

The data I currently have amounts to less than 400 Gb but would increase by about 100/200 Gb per annum.

Money is not really an object (within reason). I have three PC's on the network total - 2 pc's and a laptop. I'd like to include the other two in the backup strategy as well.

For the record, the PC is an ABIT AB9 motherboard with a core 2 duo processor. It can support 6 internal SATA drives and 2 external SATA drives. 4 Gb RAM and a 256mb Nvidia graphics card. 630W PSU.

Cheers
Dave

Dave,
I'm the option 1 type of guy - PCI card RAID controller. 5 years and no losses, thats a pretty good statistic. I will not let any piece of RAID software of that kind control my data storage.

But, also think of it as the lazy way of doing things. You just can't beat having 2 seperate locations, one backing up the other, like an external hardrive, or a second PC in your home connected wirelessly to your main machine. I also have this set up. Think of it like this, the chances of one machine being fried by lightening or a power surge is, of course, 100%. Total loss and an expensive attempt at data retrieval. But the chances of that happening to 2 machines simultaneously (for various hardware reasons), is not 100%.

Make you think?

Seanie.

I agree with Seanie. Two seperate locations seperated my a decent distance is the best option. As for hardware vs software raid systems, ive only ever had exprience with hardware managed implementations. From hearing your story I would be inclined to think software solutions are not the best route for me in future should the option arise.

As an additional precaution, is it possible that your data is stored in a database managment system of some type ( sql server, oracle mysql etc)? if so, it would be a nice extra safeguard against dataloss. Most db systems have some form of compression or can get addons ( such as lightspeed for SQL server) that allow backups to be compressed considerably, all assuming you are using a database.

If your not using a database, but flat files of some description, why not buy a backup application that will allow you to do one giant compressed backup of all data, and then a diff backup from there on in. I know this doesnt solve your current raid issue, but it would add a little piece of mind that your data is safe in the event of a total hardware failure ( of both your raid servers failing).

Thanks lads - that's great feedback.

I'm going to cost up the harware RAID (I think a 16 drive card will cost me about 600 notes) plus the cost of a couple of external drives for backups. Perhaps the NAS is the way to go for that particular setup (i.e. the main PC covered by hardware RAID and backups done remotely). The NAS systems I've been looking at, whilst expensive, seem to come with comprehensive backup software.

As a bit of an aside, and know this might be contentious, how aboutI rebuild my system with Vista? Now that SP1 is out (and I think SP2 is imminent) is it now considered stable enough? Or should I stick with XP pro...

Thanks for taking the time to reply - I really appreciate it.

Dave

I've been using Vista now for the past year and haven't had any significant issues with it. I'm not what you might term a 'power user' but it seems stable enough to me.

Phil.

I use vista on all of my computers ( desktop, dell laptop and Macbook Pro) and it works fine on all. they are all relatively high spec machines, and i wouldnt dare running vista on anything less than 2ghz , 4gb ram ( ive tried it on an old laptop and its brutal). I dont know really of any serious reasons to go from Pro to vista other than there will be (in theory) less updates weekly for vista and its slightly more secure ( google that for many good debates)

Personally I'd use Solaris and the ZFS filesystem on whatever disks I have lying around; its available on Linux and BSD now too I believe. A windows box could then access it through samba or iSCSI. All this RAID talk is a bit 1990's.

Another option that I'm looking at for my photos (currently all on DVD's all over the house...) is Amazon S3. 'The network is the computer' remember
aws.amazon.com/s3
$0.180 per GB – first 50 TB / month of storage used
$0.100 per GB – all data transfer in
$0.170 per GB – first 10 TB / month data transfer out

When you add up the cost of a few drives that starts to look good value for backup storage. There are several sites that deal with how to use this service as an automatic backup.

So please don't tell me to get a large USB drive and remember to backup to it

Use multiple usb drives and ZFS ok I'll go back to work now.

blogs.sun.com/bonwick/entry/raid_z might be of interest to techies here regarding raid and zfs.

As a bit of an aside, and know this might be contentious, how aboutI rebuild my system with Vista? Now that SP1 is out (and I think SP2 is imminent) is it now considered stable enough? Or should I stick with XP pro...

My opinions of Vista and XP aside... is something not working in XP that you need vista for? XP pro is still supported so I wouldn't change unless I had to.

Well, I just got the invoice from the recovery Company. Thank God I've got VHI cover because my heart rate approached 300 bpm and my blood pressure was enough to set off our smoke detector. Jeez - that was one very very expensive crash. Its cost me than 150% of the price of the (high-end) PC... Credit crunch? My bum - data crunch here in Celbridge...

Anyhow, I've ordered a bunch of hard disks and a NAS box plus Vista. The data recovery folks recommended this approach over a RAID card

I'll send the two hard disks back to Samsung and hopefully they'll refund me the money - small potatoes compared to the recovery costs.

I'm really *that* familiar with windows that delving into other OS's fills me with paralising fear. But the on-line backup option seems cheap for storage but I'd be interested in how muc per month the data transfers would cost.

Given that the aim of this exercise was to firstly recover the data then secondly to put in place a system to deal with a winter's worth of FITS files, I'd be worried about those transfer costs.

So, I'm back to my software RAID system plus a 2TB NAS.

Next question - for a 6 disk software RAID system, what flavour of RAID would you folks recommend? I was running RAID 1+0 which seems to give the best protection (the recovery company said that if I was running anything else, I'd've lost everything)...

Finally - Al, given this would be a complete rebuild, is the system you suggest easy andcan the stpes be written down that even an idiot (like me) could follow? I won't rule out your suggestion but it would have to be easy and Windows compatible (at the end-user level).

Cheers and thanks again for your help.

Dave

dmcdona wrote:Next question - for a 6 disk software RAID system, what flavour of RAID would you folks recommend? I was running RAID 1+0 which seems to give the best protection (the recovery company said that if I was running anything else, I'd've lost everything)...

1+0 is probably best; its expensive in terms of disks but statistically safer than 0+1 & 5 if a second disk is to go. Having 2 disks take out a 1+0 is very unlucky!

Finally - Al, given this would be a complete rebuild, is the system you suggest easy andcan the stpes be written down that even an idiot (like me) could follow? I won't rule out your suggestion but it would have to be easy and Windows compatible (at the end-user level).

I think if you are very reliant on windows and want something simple then zfs probably isn't the best. The technology is great, but you'll need to install an OS you're not familiar with and administer it. From the sounds of it thats an extra level of complexity you don't want.

Thanks Al - that's great advice.

As it happens, Komplett are doing very good deals on SATA drives at the mo - 55 quid for a 500 GB so its not too expensive.

I'd really love to try out other OS's bu at this stage of my life, I'm so used to MS that I'd really be lost without it. I remember Windows 3.1! In fact, I remember my first hard disk - a whopping 40 Mb. And even before that, I remeber using DOS and using two floppy drives (A and B ) to load it every time you wanted to use it. Cripes, does that show my age...

Thanks for all the feedback, very much appreciated.

Dave

just bring this up again... I've been using s3backup which is a little tool that uses amazon s3. Its ok. It does the job but it could be better.

A couple of other people I know have suggested this www.getdropbox.com/ havent tried it yet but it looks promising. I also havent seen the costs yet.

Al,

As an amazon account holder, I found it very easy to sign up for the Anazon Simple Storage S3.
However, for the life of me, I can not figure out how to use it.
To a not-so-techie person like me, they certainly do not make it easy to understand.

I have seen stories that google may offer an online backup system. Have you heard anything about that?

Michael.

I just downloaded that Dropbox utility. Very nice! It allows you to make a folder on your pc. You then just drag the files you want to backup in there and they automatically hget backed up onto the Dropbox server. They are then accessible anywhere provided you have access to log on (you create an account). FWIW, the Dropbox server uses the Amazon S3 servers. It also allows you to share a file to the net very easily. This would make it much easier to share files without the need for a hosting account. Storage space of 2GB is free. If one needs more space, one can get 50GB for $9.99 per month or $99 per annum. Maximum file size permitted is 350MB.
Nice!

Michael.

michaeloconnell wrote:

Al,

As an amazon account holder, I found it very easy to sign up for the Anazon Simple Storage S3.
However, for the life of me, I can not figure out how to use it.
To a not-so-techie person like me, they certainly do not make it easy to understand.

I have seen stories that google may offer an online backup system. Have you heard anything about that?

Michael.

Amazon s3 is aimed at web 2.0 developers not folks at home. They don't do a good job of telling you that. So if you are shutterbug or some video/picture online gallery site then you can write the code to use it. If you are just a home user you need to get some software from somewhere to do it for you. The software available is improving but its not really ready yet for everyone to use I dont think.

I've heard several rumours of several companies possibly offering online storage facilities

Its a real pity that you cant mirror drives in windows XP, that would be a real nice and simple way of have reasonable redundancy in the system.
Well it turns out the feature is in there, but just turned off!. You can do a small bit of hacking to get it working
Follow the instructions in the link, its quite simple but you must be extremely careful that you edit the dll and exe files exactly as they describe. The link talks about Raid5, but I'll stick to raid1, simple mirroring. This work around seems to enable w2k server mirror functionality.
www.tomshardware.com/reviews/windowsxp-m...-5-happen,925-2.html

It worked for me with 2 40gig drives, so I'm off now to get 2 terrabyte+ drives, so unless the machine gets hit by lightening, then it'll do fine.
I haven't seen a performance hit on the machine doing this, then again I dont do anything more taxing then photoshop on it. .

For those who might be in the know, mirroring is when you have 2 hard drives, where one is always an exact clone of the other, windows maintains the link, so one drive can go bang leaving you to fall back on the other, the odds of loosing the 2 simultaneously from a hardware fault are fairly low.

Turns out the solution above is only stable for XP with SP2,
for SP3 use the following.
www.optimiz3.com/low-cost-and-reliable-n...-jbod-raid-0-1-or-5/
I'm currently mirroring 2 1.5TB drives at the moment, I reckon its gonna take nearly 2 days to run

Once its done though Dave, its done... A bit like formatting a brand new hard drive...

yea, and I dont see much of a performance differance now that its up and running, no cpu cost..
It was finished when I got home today, its a little more piece of mind knowing that my mp3/pics/whatever are a little bit more secure now.

Dave - my PC crashed last wednesday. I tried to get it to work by using the "Vista repair" option on the install DVD. I wish I'd just gone back to my backup software. I started it this morning and by the time I came home, the PC was back to where it was when it crashed (I do a daily backup of all 3 pc's on the home network).

Now, as it happens, the registry was corrupt and I restoreed a "corrupt" version of it. So I've had to go through the motions again. However, I've done this knowing that I didn't lose any data at all. Not a position I've been in before and very glad I spent the money to get here.

When you have the comfort of knowing a crash will result in no or very very little data loss, you can restore at leisure.

Believe me, its a much bigger piece of mind than you can imagine - nut of course you'll only know when you crash and burn and disk or install an errant driver of program...

Dave

oh man, over the past 15 years or so I've lost many a C drive, so what I'll be doing soon is making a clone (using ghost) of the C drives on all my PCs and laptops, now that I have the space to store these clones.
If I drop a C drive, all I need do is replace the drive, boot up using my bootable usb stick, restore the drive using ghost and I'll have my C back the way it was.

Lads,
I have to say,
I'm worried.
I have 2 laptops.
I keep nothing of value except programes on them.
One is portable one is hooked up to an old Del Monitor (for image processing)
I store everything...
astro photos, mp3's, family stuff...
all on an external hard dive that I can plug into either.
The drive came with a recovery guarentee so ..
I hope if it ever failed we could get our stuff back.

ALTHOUGH..
I'm now thinking we need to back up our back up.
(ITMS)
I really can't follow what you guys are at with drives this
and ghosts that.
I think it might be ok if we just back up the back up.
Because then we would have some hope of pulling everything back if there was a crash.

Frank - perhaps you could use the hard-disks in the laptop(s) to store all your "work in progress" and use the external device as a backup drive. Essentialy, just take the external drive and divide it up either into two volumes (laptop 1 and laptop 2) or two folders (same names) and simply plug it into laptop 1, backup, then laptop 2 and backup. The supplied Windows backup software should be sufficient. I'd recommend backing up daily if you use the laptops on a daily basis.

This scheme won't cost you a penny and "should" allow you to restore either laptop, including all its software.

An alternative scheme would be to buy a second external drive and simply copy the primary external drive to your new one (again, daily would be best).

What you should do anyhow is make sure your backup can be run so that it will restore either laptop, including software installations. Though, fire away and just backup data files (e.g. docs, images etc) but be prepared to re-install all your software from scratch...

I'm not sure what would be the best software to use - personally, I use Retrospect from EM2 (got it "free" with the NAS drive). I'm pretty sure the supplied windows software would do a good enough job though.

One other word of warning - do not assume "programmes" are of no intrinsic value... For things like Photoshop and astro software, there's a pretty large investement in setting it up the way you like it. If you have to re-install, it can be a real pain inputting all those settings from scratch. Take it from me, its a lesson I've learned from bitter experience...

Finally - you have a "recovery gurantee"? Not sure what that means - but if a drive crashes (in a physically damaged kind of way), its, err, crashed and only by paying a shed load of money to an expert, can you hope to get it back. Sure, if its a software glitch, its simply a pain in the bum and you'll miss quite a few dark nights...

HTH

Dave

Dave,
thanks for the advice.
I'll mull it over and make sure to have some
kind of back up in place soon.

Barberskum wrote:

The drive came with a recovery guarentee so ..
I hope if it ever failed we could get our stuff back.

I seriously doubt it, I'd imagine what that means is that you might be able to get the laptop back the way it was when you bought it, maybe from a EISA partition on the drive, presuming windows is corrupt with no hardware problems.
If you get a mechanical fault in the drive, maybe your warrenty will cover replacing the drive and getting windows up and running, presuming the warrenty hasnt expired.
I'd seriously doubt they'd cover recovering data from a blown drive.

It was the external drive that had the guarentee.
I must re check it but unless I read it wrong it
said that if the drive failed due to a product fault
(obviously a power surge, fall or virus etc is not covered)
then they would recover the data as best they could.
Now I understand that there are some glaringly obvious issues here
and I really do need to have a better back up system.

Using the drives in the laptops is a bit of a pain as I have to
literally download new images from the camera into a folder
and then do all the work etc in Photoshop and then transfer the finished
work in the folder onto the external drive.
Then if I ever need to rework an image or pull it for printing / emailing
I take it off the External drive.

I think I need a new desktop PC that has storage and processing power
to work on stuff and then just use the external HD as the back up drive for that.
Boo...
This means spending money...
Either that or I just buy a second external drive - 1Tb or so,
seeing as they are very cheap now, and use that instead...
??