Long-term data retention

Discussion in 'General Technical Discussion' started by joelincoln, Sep 8, 2008.

  1. I'd like to get a feel for what other photogs are doing to ensure that their digital images are available in the long-term. And by long-term, I mean decades or longer.

    Obviously, backing-up data is vital and this assumes the use of stable media and standard formats.

    But...

    In 20 years, when you want that picture of Grandma when she was 60, what will you do?

    1- Will the media that you've used to store the image file (CD, DVD, etc) have survived the passing of years?

    2- Will the hardware in the future be able to read your media?

    3- Will the software in the future understand and properly render your image file along with the metadata?

    4- Would it be better to print everything and rely on the durability of the ink/paper to hold the image.

    5- Should you expect to have to "refresh" your images periodically. That is, convert them into new formats and new media every 20 years or so?

    Anyone have insight into this issue???
     
  2. Let me take a crack at this.

    Probably not. You will most likely have to transfer to new generation of media, as technology keeps advancing. How do you read the 8" floppy disk from 25 years ago? If they are still readable, that is... Then, the 5 1/4" floppies and even the 3 1/2" ones.

    Provided you can find the old hardware in working condition and the old media is not eg demagnitized, yes. But the current hardware of the far future is not likely to read old media. Transferring to current technology is probably the way to go. I rely on hard-disks for that and I'd rather avoid anything else... Progress in those is incremental (as opposed to very steep improvements in eg optical media), so you can nearly always chose when to to the transfer, most appropriately.

    This is a no-brainer. Yes, there will be software capable of rendering images, including metadata, if for no other reason because of the sheer volume of images that would otherwise be lost. There will always be a financial incentive for somebody to carry forward this rendering. You didn't ask and I didn't say that this would be provided for free! :wink:

    If you assume that we won't become a society where energy is so rare and expensive that computer technology would no longer be available to the average citizen (and the average person could not longer fly or even drive a personal vehicle that is not human or animal propelled), there will be some kind of device to show pictures. The challenge will be to determine which pictures to show: how do you pick that shot of grandma amongst the tens and tens of thousands of now-meaningless flowers, bugs, landscapes, cats, dogs, etc...? For those truly special images, having them in print may really be the way to go.

    I think so. Not necessarily converting them to a new file format, but at least converting them to current media, i.e. copying them to new drives, etc... I expect to do that sooner than 20 years, say every 3-5 years when I upgrade my computer. I carry along the data that I want to keep, converting it to those newer drives in the process.

    Not truly an insight, just some thoughts... I may be totally wrong! :eek: :biggrin:
     
  3. Interesting question and good answers, Philippe. This is an issue I have thought about a good bit, and have essentially come to the same conclusion as you; I store my files on hard disk with backup, and move to a new disk periodically.

    But in addition, I use an online backup service in case of something like a house fire, etc. So, being a nervous Nelly, I also worry occasionally about the backup service going belly-up and losing all my files.

    Nothing is certain in life but change, so backup redundancy is probably the safest bet.

    But I hate to admit how many GB I have wasted on images that are sheer junk! :redface:
     
  4. ChrisA

    ChrisA Guest

    All you have to do is design a system that meet a two basic rules and you'll be OK.

    1) There should always be at least three copies of the data (even while the backup operation is in progress, so in effect this means four not three)

    2) The data should always exist in at least two geographical locations. This for most people means rotating a backup set to an off site locations.

    In the course of doing the above you will have some media die of old age, lost, fail, stolen or catch fire or simply go out of style or become to small. You will replace it. You will replace the computer too,many times As you replace it the media will always be "current" and usable.

    Practically you would have copy #1 on your computer, copy #2 on your desktop, perhaps this would be inside Apple's "Time machine" then you'd have copy #3 at your office or your kid's or parent's house and then you'd have another copy that you keep in a fire safe and rotate with the off-site copy.

    The thing about off-site is that you can never bring it home without first bringing a the replacement to the off site location. So you need at least two drive to support and off-site backup. The most common time to loose data is during a backup operation. If lightening strikes (literally) while a backup is ongoing you still want to have redundant copies of the data. The good news is the drives are cheap. I just bought a 1TB Seagate for $160.

    I've said many times that in 50 years there will be very, very few 50 year old photos as almost no one does the above
     
  5. Good input guys. I agree with, and practice, much of what you all state.

    I don't have a big problem with the idea of transitioning media periodically as new media becomes available.

    But a larger concern, and a potentially much bigger headache, is file type compatability. Philippe makes a good arguement as to why future software will be able to render current file formats. Certainly, formats like jpeg are likely to be around for a long time due to the vast amount of images in that format.

    But what about NEF, TIFF, PSD, etc? What happens when Nikon and Adobe and Canon are all dust, and we're all using holographic cameras? Will "Holoshop version 8.2" :biggrin: understand a NEF and do a reasonable job of rendering/printing it?

    Are there any file formats that are not a good idea to use for long-term storage?
     
  6. There should be no worries about reading NEF, JPEG, PNG, TIFF and PSD (which is a variant of TIFF) in the future, even if the solution is not from Adobe or Nikon.

    Today, there is Open Source software available (including source code, of course) to perform those functions. For instance the program dcraw produces TIFF files from NEF images. Hundreds of developers are using this code, resulting in several commercial packages (for example Bibble). Dcraw is also used in larger projects, such as the free image editor Gimp (also an Open Source project).

    Anybody with access to Open Source software source code (and the Internet preserves nearly everything) could in theory continue development, adapting it to the operating system of the day, 100 years from now, and be able to read/write those file formats. Whether for free (as some of these packages are today) or not is another question...
     
  1. This site uses cookies to help personalise content, tailor your experience and to keep you logged in if you register.
    By continuing to use this site, you are consenting to our use of cookies.