HDD (or SSD) configuration for DVD/Blu-Ray ripping/encoding rig

[deadlift1]

I've got a friend who owns an A/V install company and he has asked me to build him a rig dedicated to ripping and encoding his clients DVD and Blu-Ray collections. I built one of these a few years ago as a side project and a proof of concept build. It had 7 DVD drives that were ripping DVDs to a simple HDD raid0 config.

This new build will need to focus on speed. His first client has a collection of 500 DVDs/Blu Rays and that is going to take a serious amount of time. I understand my optical drive will be the biggest bottleneck but are there any suggestions for setting up the HDD's that might speed up the process?

As things stand I am considering the following:

For ripping media - 2x 3TB drives in raid0

For encoding media - 2 x 1TB SSD raid0 or possibly 4 x 512GB SSD

Does a raid card make sense for either config?

We will be using either DVD decrypter or Any DVD HD for ripping and Handbrake for encoding.

Any insights or advice would be great.

 

[marsboer]

6TB is nothing if you are going to rip 500 movies, many of which are blu-rays, unless you are going to strip away everything and encode the movies to crap. I can't really see why someone would go through the pain of ripping their entire collection and not selecting at least lossless mkv-remuxes with only the needed stuff as a minimum. If you encode to lossy, you may have to do it again if he decide to have the optimal experience with archival quality. As ripping is such a massive pain to do again I would seriuosly consider at least some level of redundant RAID. RAID5 is immensely better than RAID0 data safety wise, even though it is not optimal with these disk sizes.

If you use 25GB as average size for a stripped down mkv-remux, you would at least ned 12.5TB in the worst case scenario of 500 blu-rays.

 

[GeorgeHR]

I do a reasonable amount of ripping. I usually rip DVDs and encode them to 4GB per movie. The quality is good enough to watch. But you should have your client determine what is good enough.

---

I would suggest more than 1 DVD/Blu-Ray drive. That is going to be the bottle neck. If you have software the can rip several disks at once things will go faster. Perhaps DVD Shrink and DVD decrypter.

I don't think RAID will help much. Set up a data set for each ripping "stream." A SSD to rip to. Recode to that SSD also. And a HDD to store the result.

 

[deadlift1]

6TB is nothing if you are going to rip 500 movies, many of which are blu-rays, unless you are going to strip away everything and encode the movies to crap. I can't really see why someone would go through the pain of ripping their entire collection and not selecting at least lossless mkv-remuxes with only the needed stuff as a minimum. If you encode to lossy, you may have to do it again if he decide to have the optimal experience with archival quality. As ripping is such a massive pain to do again I would seriuosly consider at least some level of redundant RAID. RAID5 is immensely better than RAID0 data safety wise, even though it is not optimal with these disk sizes.

If you use 25GB as average size for a stripped down mkv-remux, you would at least ned 12.5TB in the worst case scenario of 500 blu-rays.

Of the 500 discs he's only going to be ripping and encoding about 10 per day which is why it won't need a ton of storage. He'll just delete all the ones that have been completed and start up again the following day.

 

[deadlift1]

I do a reasonable amount of ripping. I usually rip DVDs and encode them to 4GB per movie. The quality is good enough to watch. But you should have your client determine what is good enough.

---

I would suggest more than 1 DVD/Blu-Ray drive. That is going to be the bottle neck. If you have software the can rip several disks at once things will go faster. Perhaps DVD Shrink and DVD decrypter.

I don't think RAID will help much. Set up a data set for each ripping "stream." A SSD to rip to. Recode to that SSD also. And a HDD to store the result.

I am definitely going to utilize multiple BD ROM drives, probably 5-7. With that being said is an SSD overkill for ripping to? I can't imagine the rip speed from the BD being anywhere close to an SSD write speeds. I do like your idea of one dedicated drive per ripping stream though.

 

[GeorgeHR]

I am definitely going to utilize multiple BD ROM drives, probably 5-7. With that being said is an SSD overkill for ripping to? I can't imagine the rip speed from the BD being anywhere close to an SSD write speeds. I do like your idea of one dedicated drive per ripping stream though.

The ripping is not going to matter. But if you are doing recoding, it might be best to limit the number of streams on a SSD.

 

[staticlag]

For encoding you don't need SSD speed.

If you are doing a quality encode you might be writing data at like 8-10MB/s.

I would not waste budget on SSDs, rather I would buy an awesome $400+ GPU for GPU encoding, that should significantly increase encoding speed.

 

[deadlift1]

For encoding you don't need SSD speed.

If you are doing a quality encode you might be writing data at like 8-10MB/s.

I would not waste budget on SSDs, rather I would buy an awesome $400+ GPU for GPU encoding, that should significantly increase encoding speed.

Everything I've read about GPU encoding shows much faster speeds but much lower quality video as a finished product. I would love to go this route for the speed alone, or intel quicsync for that matter, but the quality is just not there.

Looks like mechanical HDDs for the rig then and no need for raid.

 

[UhClem]

Looks like mechanical HDDs for the rig then and no need for raid.

For multiple (concurrent) ripping/processing sessions, (even) a single SSD may prove faster. You want to eliminate disk access time, which is where SSDs shine.

 

[gantux]

The ripping is not going to matter. But if you are doing recoding, it might be best to limit the number of streams on a SSD.

I think ripping with 5 or 7 drives is going to matter. BD single speed is 4,5 MB/s. So with only ripping at 4x (=18 MB/s) on 7 drives you are hitting your rip storage with 126 MB/s. But you are writing to 7 files simultaneously. I doubt a non-SSD drive can handle this (somewhat random) load.
Things are getting worse if you try to read from your rip storage at the same time for encoding. Or trying to move away data to a dedicated encoding storage which is basically the same.

Instead of throwing tons of money (multiple large SSDs, SATA controller for the BDs, large number of BDs, RAID, HDDs and stuff) at this rig, I would try to find a reasonably balanced setup which can be saturated by the real bottleneck - the human operator.

I'm assuming there is no kind of automatic disc changing mechanism involved, so somebody has to do it manually. As mentioned by marsboer 25GB/movie (probably without extras, comments and crap) seems like a good number to start. Depending on the drive speed (up to 16x but not realistic for the entire disc ), disc condition (scratches etc.) lets pick a conservative value of 5x (22,5 MB/s). That leads to a pure ripping time of ~ 18 min. With no automation in your software workflow (starting the ripping, language and subtitle selection, then starting encoding, maybe manual folder creation etc.) this gives us roughly 2-3 movies/hour and drive.

So with only 2 drives you are able to rip the aforementioned 10 movies a day by only occasionally dropping by the machine, changing discs and doing all settings, configuration and so completely by hand within a few minutes.

As I have no recent encoding experience regarding Blu-rays, I can't make any assumptions of how long the encoding process may take. Depending on the settings and computing power (number of cores and your software's capabilities to use more than one) this might take longer than the actual ripping process. So piling up ripped movies without the ability to encode in time won't help either.

 

[deadlift1]

I played around with MakeMKV this weekend and I think this is the route we are going to take. It eliminates the need to transcode the video to another format and provides a near lossless copy of the original source in a smaller file. It is a memory hog but that is easy to overcome and it is not cpu intensive at all. I am thinking of a large SSD Raid array with ability to rip 10 discs concurrently. The files will then be transferred to the clients hard drives via a hot swap bay.