RAID setup?

[JET]

I prefer it to run Win7 as right now I am using media player to send files to a PS3 and I also have a HTPC that sees the most playback. I am curious about the WDTV, if that can take the place of the PS3 then I am open to that.

HTPC runs Win7 AMD phenom 2 @ 3.8ghz with 4GB ram and a 128GB SSD.

The computer downstairs stores all the files, but it also used as a desktop. It is an i5 3570k, 16GB ram, 12 TB of disk space once my red's get here, etc.

All the movies are HD, with most files being 8-24GB per movie. Everything is hardwired on a gigabit network.

All of the file sharing works great, except many of the new movies have Cinavia protection and the PS3 has issues playing them. The HTPC has no issue with them.

[bramagedained]

http://en.wikipedia.org/wiki/Standard_RAID_levels

Running just the disks as standalones and doing regular backups might be your best bet.

It sounds like you have too much storage for RAID 1 to be reasonable because it's too inefficient.

RAID 5 is popular, but, there is some concern about recovering from a disk failures now that disks are so big. As your odds of bad sectors among big disks is reasonably high. Data density has increased a lot in the past few years, but, the number of bad sectors/GB(or sectors that will fail over the life of a drive) hasn't really gone down a whole lot.

The 'cool' setup is what a friend of mine did, and likely what I will copy when I get around to making an actual file server. He runs a dedicated RAID controller that takes up a PCI slot with (I think) 10 2TB disks connected running a RAID 6 setup. Its really fast and can recover from multiple drive failures. It's probably somewhat excessive for home use. We had a couple of gigabit LANs setup with ~30 users in the barracks that pulled from it for streaming movies and such.

I don't/haven't worked in a datacenter or anything just what I've delt with for some of the government systems I've supported in the past where people would(did) get really angry if the data was lost.

[19TsiAwd90]

Yeah, I run a 128GB SSD in the HTPC upstairs and it took about 15 sec of boot time off. It boots in 21 seconds now. I have been checking out Peter's suggestion of using Windows to run a RAID 1, which just mirrors the drives. I didn't even know Win7 would do that. Thanks Peter!

face palm raid1 is what i was refering to not raid 10..

[scheides]

What do you use the PS3 for versus HTPC? Like what are some of the end goals you have for the whole setup? Why are your files SO huge? You'd be surprised at the quality of a nicely ripped/encoded 1.4GB movie file and even most blue-rays are under 5GB uncompressed. 8+GB seems just redonk man. Like I said I have a total of 1TB of storage and I have all seasons of Top Gear and a bunch of other tv shows, at least a hundred movies, and off-site backups of mitsystyle.com, scheides.com, etc.

Check out the WDTV or better yet ask Spazzy about it since he's around you a bit more...it's so badass for the money! All I use it for is watching um, legal copies of dvds and blue-rays I already own, and youtube (now youtube 'leanback'). It has netflix support and hulu plus as well, all in full 1080p.

http://www.wdc.com/en/products/products.aspx?id=330

I actually have the 'WD TV Live Plus' but I purchased it almost 3 years ago...I think the most basic model now does all the same stuff but is physically slightly smaller.


Don't forget that with RAID you loose some percentage of total disk space, that's the trade-off for redundancy. RAID6 is a great option but you gotta really pony up to make the investment for a bunch of similar/identical disks and upgrading is gonna just as painful down the road.

[scheides]

Also, just because I am all about KISS, you can REALLY do an out-of-the-box home entertainment setup with a WDTV and one of these:

http://www.amazon.com/Personal-Cloud-Storage-Share-Photos/dp/B00AGI7I2O

They offer a 4, 6, and 8TB model...all do RAID1 (as far as I can tell). The 8TB model is only $625 which is actually a pretty screaming deal...the drives are user-serviceable too so if one of them takes a digger you can just swap it out yourself.

*edit* 8TB is in JBOD mode, the 8TB model would be 4TB in RAID1 (as far as I can tell).

Add in data your favorite torrent site or some ripped media from any laptop and you're golden.

[JET]

The PS3 is just used as a media player because I don't have a 2nd HTPC. My files are so huge because I only play HD movies at full frame rate. 18+GB is full bluray quality. This part I do know a lot about, check the frame rate on the supposed 1080p movies if they are under 5GB. I play them on a 50" and anything under 3GB looks like crap. You can also tell the difference between a 8GB file and a 16GB, just like an upconverted DVD vs a Bluray.

I'll have to explore the set top boxes some more, it looks like there are 17 models out there now.

It looks like RAID 1 would be the best option, but I think I might just get a JBOD box and just do a backup once a month. With RAID 1 I wouldn't be able to unplug the backed up drives, would I? I only want them running when I do a backup once a month. I don't trust the reliability of the seagate drives to stay on all the time.

[scheides]

Hah well I guess not being a 'videophile' is worth it for me, dealing with that much data just seems like a huge PITA. Then again that's what I do on a daily basis and anything less than a 200TB infrastructure-grade disk array is just janky basement shit to me (not worth spending much of my free time on). :)

In any raid set you need to leave them all running all the time or you will be wasting your time scrubbing and frankly risk the other disk failing. You will also loose the read performance bennifit which frankly you may need to rely upon if you insist at pumping that much data; a single disk may not be able to keep up. You will need a whole extra drive to back your main storage pool up.

Keep in mind with JBOD (no raid) you will have to rebuild the entire array if any of the disks fail. Just like cars, do it right, do it once, more movies and less 'wrenching' time.

[JET]

But if I run no RAID and use just JBOD, I would just have to restore the one drive that fails if I use the same size drives in both the PC and the JBOD, right? As I understand it is basically a bunch of external HD's in one enclosure.

I am beginning to think I should just use one of these on my eSATA port and pop my backup HD's on whenever I want to do a backup.

http://www.newegg.com/Product/Product.aspx?Item=N82E16817153112

That is exactly why I started this thread, I was going around between the options and couldn't really figure out which one was the best.

[scheides]

A true JBOD will take 'a bunch of disks' and present logical volumes to the network, with not a ton of regard to what it puts where. That's why I mentioned the warning about huge and lengthy restore jobs.

Now that being said, with what you have described you have a win7 machine that you use for a desktop or whatever...throw disks in it and share them out individually. Add the latest and greatest 'big' single disk that meets your budget when you need more space. File server, done.

For backups, I use an almost identical enclosure to the one you linked. When you buy a big disk above, budget for a second one the same size or bigger and dump your files to it (and your other secondary disks that match your other drives) and store in a safe, dry place.

Write some bat scripts (or better yet bash after you install Cygwin) and make the process easy on yourself.

This is a pretty simple setup but still plenty of room for failure and still not sure about I/O performance. Let me know what you end up doing!

[JET]

Alright, that sounds like the best option then. The drives only run when I have them on the dock and I can leave the dock hooked up all the time. I could honestly probably just do a copy from the main drive to the backup and tell it not to copy the files that are already there. There will only be about 300 files per drive max.