1. Welcome to NikonCafe.com—a friendly Nikon camera & photography discussion forum!

    If you are thinking of buying a camera or need help with your photos, you will find our forum members full of advice! Click here to join for free!

RAID Questions

Discussion in 'Apple/Mac' started by nu2scene, Sep 7, 2018.

  1. nu2scene

    nu2scene

    Sep 27, 2007
    Las Vegas
    I'm currently doing some reading on RAIDs. I think I have a very general overall idea of the concepts, but as I'm reading I'm coming up with some questions. I'm hoping there are some people that can help.

    I'm thinking of getting a g-tech g-raid drive to store my images on instead of an internal drive. I also want to edit the images off the raid drive. I'm thinking of running it in raid0, so it's faster. I know it's not redundant in raid0, but I figure it's the same as if I was running the images off a regular external drive. If the raid0 fails I lose everything, but if a regular external drive fails, I also lose everything, so it's the same situation. Unless I'm misssing something?

    I will back up the raid0 in case it fails, just like I currently have my images on a regular external drive and have a backup of that drive.

    My questions are:
    1. Is my thinking flawed? Or am I on the right track?
    2. I know a raid0 transfers info faster than a regular drive, but will I see a real world difference in PS and LR vs just using one 7200rpm drive to edit off of?
    3. For the back up of the raid0, I'm wondering if backing up to a regular drive is safer, as far as corruption or some sort of failure vs running a raid1 for the back up. I was thinking running a raid1 would then give me 2 back ups instead of just one.
    3. Would a raid1 backup of the raid0 be less safe than having 2 separate external drives, or is it a wash?
    4. With a raid1 setup, if one of the drives fails, does the entire drive stop working until you put in another working drive, or does the one working drive continue going?
    5. Is there anything else I should be considering?

    Thanks for your help.
     
    Last edited: Sep 7, 2018
  2. Hey Rob,

    1. No, your thought process is correct. Having 3 copies of your data (preferably with 1 stored at a separate geographical location) provides great protection from data loss and each pair of striped drives (RAID 0) in an enclosure can be considered 1 backup in the overall backup strategy.
    2. Without knowing your system specifics, the type of drives that you might use (or G-tech provides) in the device, or how it connects to the computer that you're editing from, it's hard to know exactly how great of a speed benefit you'll see.
    3. You could back up to a regular drive or another pair of striped drives - I've done both and never had an issue with either solution because the operating system abstracts the 1 drive/2 drives away from your view when it reads/writes the filesystem. The one way in which a single regular drive might be better is if something with the 2nd set of striped drives breaks and the disks are ok but the enclosure/power supply fail and let's say G-tech is out of business and their filesystem is proprietary and you can't replace the enclosure that the disks are in so they can't be read from. In that somewhat unlikely scenario, your backup may not be recoverable.
    4. It depends on the hardware manufacturer. With Synology, it keeps chugging along with 1 drive and lets you know that you're down a drive and then when you replace the bad drive, it starts populating the new drive from the existing drive.
    5. Some people will go with RAID 1 with the mindset of "I have two drives that are identical so I have two backups and I don't need to do anything else", but if there's some sort of infection/file operation error/etc that damages a file, both drives will replicate the problem right away and you may be on thin ice. Additionally, if your house burns down or someone burgles your home and grabs the enclosure, two of your backups are gone, if not all 3. With that in mind, the third backup should ideally be a separate device and be stored elsewhere as I mentioned earlier and you can bring it by from time to time to sync and update the latest file changes.

    Sean
     
    Last edited: Sep 7, 2018
  3. nu2scene

    nu2scene

    Sep 27, 2007
    Las Vegas
    Hi Sean,
    Thanks for the reply! I'm still just thinking through my set up plans so I really appreciate everyones input. I will be using an online backup as well. I'm looking into Backblaze at the moment.

    The g-tech drives I'm looking at right now are thunderbolt 3 running at something like 225MB/s for the regular drives. The g-tech raids are supposed to be something like 440 MB/s. So in theory I suppose they transfer data at twice the speed. But I was just wondering if after hooking them up to an imac, would I really see much of a benefit when editing photos, or import/exporting. I'm waiting on Apple to see what their new imacs look like before I decide on any new drives.

    I didn't think about the drives using some sort of proprietery system. I know Drobo does that, but I'm not sure about g-tech. I'll have to look into it. Thanks for that suggestion.

    The g-raid I was looking at is just 2 drives. So I was wondering if I had that set up as a raid1 as backup, if one drive died, does it shut down the whole thing. Or does the one good drive still keep going on its own. If it doesn't, I would think it would make if kind of less than useful in that situation, as a backup. I'll have to look into it further.

    Thanks for the info!
     
  4. TonyW

    TonyW

    Jan 15, 2010
    UK
    There are other ways to look at this. In no particular order:

    1. Why are you considering RAID in the first place?

    2. The need for RAID in business that demands 24/7 access 365 days per year is really the reason for RAID arrays.

    3. In implementing RAID solution you should be keeping spare disks available for hot swapping.

    4. One school of thought suggests that you are now doubling your risk of failure with 2 drives effectively operating as a single drive

    5. You are adding a layer of complexity both hardware and software that you can probably do without.

    6. Etc

    Having 3 copies of your data (rotating) is great with one stored off site.
    But consider investing instead in SSD’s for speed for online/offline storage, for simplicity and reliability. Even fast spinners can work very well for storing image files if budget is tight.

    Sorry if the response not what you are hoping for but...

    Edit:
    RAID description pros and cons
    RAID level 0, 1, 5, 6 and 10 | Advantage, disadvantage, use

    Why RAID is (usually) a Terrible Idea
     
    Last edited: Sep 7, 2018
    • Agree Agree x 1
  5. And work in something in the 'cloud' for your especially important stuff. It is cheap these days.
     
  6. nu2scene

    nu2scene

    Sep 27, 2007
    Las Vegas
    The main reason I'm considering RAIDs is for increased speed. I keep my images stored on an external HD currently, and I also edit from it. I don't keep my images on the internal HD on my imac. So I was thinking I should get a quick ext HD set up for better performance. That's basically the reason, I'm considering a RAID set up. Do you feel the speed improvement with a RAID vs a regular spinner 7200 RPM HD is not worth the trouble?

    SSD's I think would be great. But I'm not sure I can justify the cost. The mobile lower density SSDs are priced ok. But the higher density SSD's seem to get pretty expensive pretty quickly. About $1000.00 per TB. I don't think I'd want to spend that much money on storage. Probably in the next few years the prices will drop into a more do-able area though.
     
  7. nu2scene

    nu2scene

    Sep 27, 2007
    Las Vegas
    Tony, that was an interesting article. Thank you for that!
     
  8. What is the data connection to the external/(and likely new raid) ?
    That will probably be the bottleneck given the large buffers in most modern drives.
     
  9. nu2scene

    nu2scene

    Sep 27, 2007
    Las Vegas
    Thunderbolt 3.
     
  10. TonyW

    TonyW

    Jan 15, 2010
    UK
    I use a PC so I am rather cautious to comment on specifics (I think you may be on Mac?) but to generalise and give an example. My PC system (several years old):
    Main drive C: Samsung SSD 850 EVO 500GB - This drive dedicated to OS and applications
    Data Drive D: Seagate ST2000DM001 2TB SATA III 7200 RPM - This drive is mainly for data although I have put a few applications here that are not critical (to leave plenty of room on OS/App drive)
    Rather loathe to mention this but I do have a QNAP RAID configured as RAID 1 with 2 4TB WD Red drives acting as data duplication mainly. I do not really need this but it was given as a gift - I could just as well copied data to seperate HDD :oops:  :LOL: 

    File display time with PS already open image data coming from the 7200 RPM spinner:
    It takes approx 3 seconds to display a 51MP, 75MB DNG file in PS
    It takes approx 1.5 seconds to display a 36MP, 38MB NEF
    Lightroom same file click thumbnail appears instantly then within couple seconds displays full res.

    I am quite happy with speed of display of images from the 7200 RPM disk. And I have not been unhappy about the speed of injesting files from either USB card reader or via camera USB.

    So for me I would say that any potential speed improvement with RAID 0 is not going to be worth the trouble. I think there may be better ways to spend your $ improving your system
    I suspect that you will only see improvement in benchmark tests but unlikely to be noticeable in real world usage
     
    Last edited: Sep 8, 2018
  11. tony-b

    tony-b

    242
    Sep 13, 2008
    Viera, FL
    I purchased an iMac in 2017 and have a WD My Book 2TB drive connected over Thunderbolt 3 to hold my primary photo library. This is backed up to a duplicate MyBook Drive and then to my Synology NAS (8 bays, RAID6 two drive vault tolerance). I haven’t found an offsite solution, yet.

    The drive is not where I see a speed issue. I see the issue more from the iMac’s processor and graphics refresh. I increased the RAM to 32GB and that works very well for photo and other applications. My processor is an i5. When I buy another, it will be minimum i7 and maybe i9 if available. Apple doesn’t give a lot of graphics options in their computers so if I stay with them, and most likely will because I hate Windows 10 in anything but the LTSB version, I’ll just go with the best they offer.

    If you upgrade the processor and graphics, I don’t think you’ll notice any drive connected over thunderbolt 3 to be an issue. They all spin at 7200 RPM. Yeah, RAIDs let you increase the speed but you can get same or better with a single SSD and less points of failure. Think about I/O needed for photo editing. Most of it is all processor, RAM and graphics card intensive. Once the image is on the drive, there’s not a lot of drive activity going on.
     
    • Like Like x 1
  12. nu2scene

    nu2scene

    Sep 27, 2007
    Las Vegas
    TonyW and Tony-b thank you to you both! That's the kind of real world experienced info I was looking for.
     
  13. Walter

    Walter

    Jan 13, 2006
    Columbia, Maryland
    Walter Rowe
    Another option multiple copies of HDDs for long term storage, and smaller more affordable SSD for active work. You can use a combo of a smaller SSD / HDD for working files, and all-larger HDDs for backups of inactive work. As you complete work, move it from SSD/HDD active drives to long term large HDDs. This gets you peace of mind that long term storage is safe while active work is speedy and still backed up.

    More RAM also helps performance a great deal.
     
  14. tenplanescrashing

    tenplanescrashing

    Oct 15, 2008
    GA
    This, to me, is a better solution than RAID. I have a boot SSD and a spinner in my laptop and do all of my photo editing from the SSD then when i'm done I move them to the spinner and update the path in LR. If I need to work on a photo from there, it's usually 1 or 2 here and there.

    I also do regular backups of both drives to external drives: one that gets stored on my desk and one in a fire safe. Next up for me is a NAS that has a cloud backup option so I can centralize files.
     
    • Agree Agree x 1
  15. nu2scene

    nu2scene

    Sep 27, 2007
    Las Vegas
    Thank you Walter and Tenplanescrashing. I haven't thought of doing that.

    So how do you do that? Transfer the images from your CF card to the ssd. Edit in LR. Then transfer the images to a HHD, and then have LR re-locate the images?
     
  16. tenplanescrashing

    tenplanescrashing

    Oct 15, 2008
    GA
    Exactly. When you physically move the files from SSD to HDD you'll get the question mark next to folder. Tell it to relocate and point to the new path and all is good.
     
  17. I think RAID is a poor choice for 99% of home users.

    I import to the internal SSD on my MBPro, edit in LR, export as catalog then import as catalog on my home system. Easy and retains all metadata (edits).
     
    • Agree Agree x 2
    • Like Like x 1
  18. tenplanescrashing

    tenplanescrashing

    Oct 15, 2008
    GA
    I've heard of people doing the export as catalog thing but I still have a hard time understanding why i'd do that per folder/shoot.

    My method keeps all of the edits as well and allows me to see all of my photos in one catalog so I can switch between them without opening a new catalog.
     
  19. The name is misleading. When you import from another catalog it adds the photos and metadata from that catalog to the catalog you have open. It has the same effect as what you are doing without the step of telling LR to find the "missing" images.
     
  20. I have my photo library (approx 650GB) and iTunes library (approx 600GB) running on RAID 0 connected by Thunderbolt 3 to my iMac and find it's more than equal, if not faster, than the test I ran with images on my internal 1TB Fusion drive. Probably because the HDD portion of Apple's Fusion is 5400rpm, vs the 3 7500 rpm drives in my RAID enclosure. I also have another 7500rpm single drive in the enclosure that's used for on site backups of my images and iTunes. My device is an older Promise Thunderbolt enclosure which is probably actually running at Thunderbolt 2 speeds.

    I did recently discover something I was not aware of ... if you have Amazon Prime, UNLIMITED storage for photos is included for FREE!! It took me several weeks to upload the entire library, but they are up there now so I have a very secure off site storage solution at no cost!!

    Ken
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.